vntl llama3 8b v2 gguf

提供商lmg-anon
分类translation
许可证llama3
下载量690.3K
星标0

简介

vntl llama3 8b v2 是一款基于 Llama3-8B 经过专门翻译能力增强的量化模型。它采用了 GGUF 格式，这意味着中国开发者可以通过 llama.cpp 或 LM Studio 等轻量化工具在个人电脑（甚至只有 CPU 的设备）上直接运行，无需昂贵的 GPU 显存。该模型旨在优化翻译质量，在保持 Llama3 原生逻辑能力的同时，提升了跨语言转换的自然度，非常适合作为本地化的翻译插件后端或构建轻量级翻译工作流，是追求隐私且对翻译质量有要求的用户的理想选择。

核心亮点

  • 基于 Llama3 8B 增强翻译能力，语言转换更自然
  • GGUF 格式支持，低显存环境下即可流畅运行
  • 适配 LM Studio 等工具，本地部署上手难度极低
  • 兼顾通用能力与翻译专项，适合构建本地翻译助手

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("lmg-anon/vntl-llama3-8b-v2-gguf")
tokenizer = AutoTokenizer.from_pretrained("lmg-anon/vntl-llama3-8b-v2-gguf")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download lmg-anon/vntl-llama3-8b-v2-gguf

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download lmg-anon/vntl-llama3-8b-v2-gguf config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('lmg-anon/vntl-llama3-8b-v2-gguf')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/lmg-anon/vntl-llama3-8b-v2-gguf

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/lmg-anon/vntl-llama3-8b-v2-gguf

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('lmg-anon/vntl-llama3-8b-v2-gguf')
tokenizer = AutoTokenizer.from_pretrained('lmg-anon/vntl-llama3-8b-v2-gguf')

完整文档

来源: HuggingFace

---
license: llama3
datasets:

  • lmg-anon/VNTL-v5-1k

language:
  • ja

  • en

base_model: rinna/llama-3-youko-8b
pipeline_tag: translation
---

Summary

This is a LLaMA 3 Youko qlora fine-tune, created using a new version of the VNTL dataset. The purpose of this fine-tune is to improve performance of LLMs at translating Japanese visual novels to English.

Unlike the previous version, this one doesn't includes the "chat mode".

Notes

For this new version of VNTL 8B, I've rebuilt and expanded VNTL's dataset from the groud up, and I'm happy to say it performs really well, outperforming the previous version when it comes to accuracy and stability, it makes far fewer mistakes than it even when running at high temperatures (though I still recommend temperature 0 for the best accuracy).

Some major changes in this version:

  • Switched to the default LLaMA3 prompt format since people had trouble with the custom one

  • Added proper support for multi-line translations (the old version only handled single lines)

  • Overall better translation accuracy

One thing to note: while the translations are more accurate, they tend to be more literal compared to the previous version.

Sampling Recommendations

For optimal results, it's highly recommended to use neutral sampling parameters (temperature 0 with no repetition penalty) when using this model.

Training Details

This fine-tune was done using similar hyperparameters as the previous version. The only difference is the dataset, which is a brand-new one.

  • Rank: 128
  • Alpha: 32
  • Effective Batch Size: 45
  • Warmup Ratio: 0.02
  • Learning Rate: 6e-5
  • Embedding Learning Rate: 1e-5
  • Optimizer: grokadamw
  • LR Schedule: cosine
  • Weight Decay: 0.01

Train Loss: 0.42

Translation Prompt

This fine-tune uses the LLaMA 3 prompt format, this is an prompt example for translation:

code
<|begin_of_text|><|start_header_id|>Metadata<|end_header_id|>


[character] Name: Uryuu Shingo (瓜生 新吾) | Gender: Male | Aliases: Onii-chan (お兄ちゃん)
[character] Name: Uryuu Sakuno (瓜生 桜乃) | Gender: Female<|eot_id|><|start_header_id|>Japanese<|end_header_id|>


[桜乃]: 『……ごめん』<|eot_id|><|start_header_id|>English<|end_header_id|>


[Sakuno]: 『... Sorry.』<|eot_id|><|start_header_id|>Japanese<|end_header_id|>


[新吾]: 「ううん、こう言っちゃなんだけど、迷子でよかったよ。桜乃は可愛いから、いろいろ心配しちゃってたんだぞ俺」<|eot_id|><|start_header_id|>English<|end_header_id|>


[Shingo]: "Nah, I know it’s weird to say this, but I’m glad you got lost. You’re so cute, Sakuno, so I was really worried about you."<|eot_id|>

The generated translation for that prompt, with temperature 0, is:

code
[Shingo]: "Nah, I know it’s weird to say this, but I’m glad you got lost. You’re so cute, Sakuno, so I was really worried about you."

Trivia

The Metadata section isn't limited to character information - you can also add trivia and teach the model the correct way to pronounce words it struggles with.

Here's an example:

code
<|begin_of_text|><|start_header_id|>Metadata<|end_header_id|>


[character] Name: Uryuu Shingo (瓜生 新吾) | Gender: Male | Aliases: Onii-chan (お兄ちゃん)
[character] Name: Uryuu Sakuno (瓜生 桜乃) | Gender: Female
[element] Name: Murasamemaru (叢雨丸) | Type: Quality<|eot_id|><|start_header_id|>Japanese<|end_header_id|>


[桜乃]: 『……ごめん』<|eot_id|><|start_header_id|>English<|end_header_id|>


[Sakuno]: 『... Sorry.』<|eot_id|><|start_header_id|>Japanese<|end_header_id|>


[新吾]: 「ううん、こう言っちゃなんだけど、迷子でよかったよ。桜乃は叢雨丸いから、いろいろ心配しちゃってたんだぞ俺」<|eot_id|><|start_header_id|>English<|end_header_id|>

The generated translation for that prompt, with temperature 0, is:

code
[Shingo]: "Nah, I know it’s not the best thing to say, but I’m glad you got lost. Sakuno’s Murasamemaru, so I was really worried about you, you know?"

查看官方来源