MiniCPM V 4 5 GGUF

提供商second-state
分类visual-question-answering
许可证Apache-2.0
下载量2.2K
星标0

简介

MiniCPM-V 2.6 (GGUF版) 是一款极具性价比的端侧多模态大模型。它最大的特点是在极小的参数规模下，实现了媲美 GPT-4V 的视觉理解能力，尤其擅长高分辨率图像分析和 OCR 文字识别。得益于 GGUF 格式，该模型通过 llama.cpp 等工具可以轻松在个人电脑甚至手机端本地运行，无需昂贵的显卡。对于需要处理私密图片、构建本地视觉助手或在资源受限环境下实现图文交互的开发者来说，这是一个理想的轻量化选择。

核心亮点

  • 端侧运行，低内存占用且响应速度快
  • 强大的高分辨率图像识别与 OCR 能力
  • 兼容 GGUF 格式，部署门槛极低
  • 适用于本地视觉助手及私密图文分析

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("second-state/MiniCPM-V-4_5-GGUF")
tokenizer = AutoTokenizer.from_pretrained("second-state/MiniCPM-V-4_5-GGUF")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download second-state/MiniCPM-V-4_5-GGUF

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download second-state/MiniCPM-V-4_5-GGUF config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('second-state/MiniCPM-V-4_5-GGUF')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/second-state/MiniCPM-V-4_5-GGUF

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/second-state/MiniCPM-V-4_5-GGUF

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('second-state/MiniCPM-V-4_5-GGUF')
tokenizer = AutoTokenizer.from_pretrained('second-state/MiniCPM-V-4_5-GGUF')

完整文档

来源: HuggingFace

---
base_model: openbmb/MiniCPM-V-4_5
model_creator: openbmb
model_name: MiniCPM-V-4_5
quantized_by: Second State Inc.
pipeline_tag: visual-question-answering
language:

  • en

  • zh

---

<!-- header start -->
<!-- 200823 -->
<div style="width: auto; margin-left: auto; margin-right: auto">
<img src="https://github.com/LlamaEdge/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;">
</div>
<hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
<!-- header end -->

MiniCPM-V-4_5-GGUF



Original Model



openbmb/MiniCPM-V-4_5

Run with LlamaEdge



  • LlamaEdge version: coming soon


<!-- - LlamaEdge version: v0.25.1 and above -->

  • Prompt template


- Prompt type: minicpmv

- Prompt string

``text
<|system|>
{system_message}<|end|>
<|user|>
{user_message_1}<|end|>
<|assistant|>
{assistant_message_1}<|end|>
<|user|>
{user_message_2}<|end|>
<|assistant|>
`

The {user_message_n} has the format: {image_base64_encoding_string}\n{user_question}.

  • Context size: 128000


  • Run as LlamaEdge service


`bash
wasmedge --dir .:. \
--nn-preload default:GGML:AUTO:MiniCPM-V-4_5-Q5_K_M.gguf \
llama-api-server.wasm \
--prompt-template minicpmv \
--ctx-size 128000 \
--llava-mmproj MiniCPM-V-4_5-mmproj-f16.gguf \
--model-name minicpmv-26
``

Quantized GGUF Models



| Name | Quant method | Bits | Size | Use case |
| ---- | ---- | ---- | ---- | ----- |
| MiniCPM-V-4_5-Q2_K.gguf | Q2_K | 2 | 3.28 GB| smallest, significant quality loss - not recommended for most purposes |
| MiniCPM-V-4_5-Q3_K_L.gguf | Q3_K_L | 3 | 4.43 GB| small, substantial quality loss |
| MiniCPM-V-4_5-Q3_K_M.gguf | Q3_K_M | 3 | 4.12 GB| very small, high quality loss |
| MiniCPM-V-4_5-Q3_K_S.gguf | Q3_K_S | 3 | 3.77 GB| very small, high quality loss |
| MiniCPM-V-4_5-Q4_0.gguf | Q4_0 | 4 | 4.77 GB| legacy; small, very high quality loss - prefer using Q3_K_M |
| MiniCPM-V-4_5-Q4_K_M.gguf | Q4_K_M | 4 | 5.03 GB| medium, balanced quality - recommended |
| MiniCPM-V-4_5-Q4_K_S.gguf | Q4_K_S | 4 | 4.80 GB| small, greater quality loss |
| MiniCPM-V-4_5-Q5_0.gguf | Q5_0 | 5 | 5.72 GB| legacy; medium, balanced quality - prefer using Q4_K_M |
| MiniCPM-V-4_5-Q5_K_M.gguf | Q5_K_M | 5 | 5.85 GB| large, very low quality loss - recommended |
| MiniCPM-V-4_5-Q5_K_S.gguf | Q5_K_S | 5 | 5.72 GB| large, low quality loss - recommended |
| MiniCPM-V-4_5-Q6_K.gguf | Q6_K | 6 | 6.72 GB| very large, extremely low quality loss |
| MiniCPM-V-4_5-Q8_0.gguf | Q8_0 | 8 | 8.71 GB| very large, extremely low quality loss - not recommended |
| MiniCPM-V-4_5-f16.gguf | f16 | 16 | 16.4 GB| |
| MiniCPM-V-4_5-mmproj-f16.gguf | f16 | 16 | 1.10 GB| |

*Quantized with llama.cpp b6301.*

查看官方来源