Qwen2.5 VL 7B Instruct

提供商Qwen
分类image-text-to-text
许可证apache-2.0
下载量5.1M
星标438

简介

Qwen2.5-VL-7B-Instruct 是阿里通义千问团队推出的新一代多模态视觉语言模型。相比前代，它在图像理解、文档解析和视频分析能力上有显著提升，尤其擅长处理高分辨率图片中的细节文字和复杂图表。对于开发者而言，它在保持 7B 参数量轻量化之余，提供了极强的指令遵循能力，能精准执行复杂的视觉分析任务。无论是构建自动化报表分析工具，还是开发智能视觉助手，它都提供了极高的性价比，且兼容主流的推理框架，上手门槛低。

核心亮点

  • 极强的 OCR 能力，精准识别复杂文档与图表
  • 支持长视频理解，可分析视频内容并生成摘要
  • 7B 参数轻量高效，适合私有化部署与端侧应用
  • 指令遵循出色，能根据视觉信息完成复杂推理

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download Qwen/Qwen2.5-VL-7B-Instruct

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download Qwen/Qwen2.5-VL-7B-Instruct config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Qwen/Qwen2.5-VL-7B-Instruct')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Qwen/Qwen2.5-VL-7B-Instruct')
tokenizer = AutoTokenizer.from_pretrained('Qwen/Qwen2.5-VL-7B-Instruct')

模型下载

我们推荐使用命令行或者 ModelScope SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 ModelScope：

操作指引 
pip install modelscope

命令行下载

下载完整模型库

下载完整模型库 
modelscope download --model Qwen/Qwen2.5-VL-7B-Instruct

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例）

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例） 
modelscope download --model Qwen/Qwen2.5-VL-7B-Instruct README.md --local_dir ./dir

更多更丰富的命令行下载选项，可参见具体文档

SDK 下载

SDK 下载 
# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Qwen/Qwen2.5-VL-7B-Instruct')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://www.modelscope.cn/Qwen/Qwen2.5-VL-7B-Instruct.git

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/Qwen/Qwen2.5-VL-7B-Instruct.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook 快速开发

下载并安装 ModelScope library

下载并安装 ModelScope library 
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

模型加载和推理

模型加载和推理 
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'Qwen/Qwen2.5-VL-7B-Instruct')

完整文档

来源: HuggingFace

---
license: apache-2.0
language:

  • en

pipeline_tag: image-text-to-text
tags:
  • multimodal

library_name: transformers
---

Qwen2.5-VL-7B-Instruct

<a href="https://chat.qwenlm.ai/" target="_blank" style="margin: 2px;"> <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/> </a>

Introduction

In the past five months since Qwen2-VL’s release, numerous developers have built new models on the Qwen2-VL vision-language models, providing us with valuable feedback. During this period, we focused on building more useful vision-language models. Today, we are excited to introduce the latest addition to the Qwen family: Qwen2.5-VL.

#### Key Enhancements:

  • Understand things visually: Qwen2.5-VL is not only proficient in recognizing common objects such as flowers, birds, fish, and insects, but it is highly capable of analyzing texts, charts, icons, graphics, and layouts within images.

  • Being agentic: Qwen2.5-VL directly plays as a visual agent that can reason and dynamically direct tools, which is capable of computer use and phone use.
  • Understanding long videos and capturing events: Qwen2.5-VL can comprehend videos of over 1 hour, and this time it has a new ability of cpaturing event by pinpointing the relevant video segments.
  • Capable of visual localization in different formats: Qwen2.5-VL can accurately localize objects in an image by generating bounding boxes or points, and it can provide stable JSON outputs for coordinates and attributes.
  • Generating structured outputs: for data like scans of invoices, forms, tables, etc. Qwen2.5-VL supports structured outputs of their contents, benefiting usages in finance, commerce, etc.

#### Model Architecture Updates:

  • Dynamic Resolution and Frame Rate Training for Video Understanding:

We extend dynamic resolution to the temporal dimension by adopting dynamic FPS sampling, enabling the model to comprehend videos at various sampling rates. Accordingly, we update mRoPE in the time dimension with IDs and absolute time alignment, enabling the model to learn temporal sequence and speed, and ultimately acquire the ability to pinpoint specific moments.

<p align="center">
<img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2.5-VL/qwen2.5vl_arc.jpeg" width="80%"/>
<p>

  • Streamlined and Efficient Vision Encoder

We enhance both training and inference speeds by strategically implementing window attention into the ViT. The ViT architecture is further optimized with SwiGLU and RMSNorm, aligning it with the structure of the Qwen2.5 LLM.

We have three models with 3, 7 and 72 billion parameters. This repo contains the instruction-tuned 7B Qwen2.5-VL model. For more information, visit our Blog and GitHub.

Evaluation

Image benchmark

| Benchmark | InternVL2.5-8B | MiniCPM-o 2.6 | GPT-4o-mini | Qwen2-VL-7B |Qwen2.5-VL-7B |
| :--- | :---: | :---: | :---: | :---: | :---: |
| MMMU<sub>val</sub> | 56 | 50.4 | 60| 54.1 | 58.6|
| MMMU-Pro<sub>val</sub> | 34.3 | - | 37.6| 30.5 | 41.0|
| DocVQA<sub>test</sub> | 93 | 93 | - | 94.5 | 95.7 |
| InfoVQA<sub>test</sub> | 77.6 | - | - |76.5 | 82.6 |
| ChartQA<sub>test</sub> | 84.8 | - |- | 83.0 |87.3 |
| TextVQA<sub>val</sub> | 79.1 | 80.1 | -| 84.3 | 84.9|
| OCRBench | 822 | 852 | 785 | 845 | 864 |
| CC_OCR | 57.7 | | | 61.6 | 77.8|
| MMStar | 62.8| | |60.7| 63.9|
| MMBench-V1.1-En<sub>test</sub> | 79.4 | 78.0 | 76.0| 80.7 | 82.6 |
| MMT-Bench<sub>test</sub> | - | - | - |63.7 |63.6 |
| MMStar | 61.5 | 57.5 | 54.8 | 60.7 |63.9 |
| MMVet<sub>GPT-4-Turbo</sub> | 54.2 | 60.0 | 66.9 | 62.0 | 67.1|
| HallBench<sub>avg</sub> | 45.2 | 48.1 | 46.1| 50.6 | 52.9|
| MathVista<sub>testmini</sub> | 58.3 | 60.6 | 52.4 | 58.2 | 68.2|
| MathVision | - | - | - | 16.3 | 25.07 |

Video Benchmarks

| Benchmark | Qwen2-VL-7B | Qwen2.5-VL-7B |
| :--- | :---: | :---: |
| MVBench | 67.0 | 69.6 |
| PerceptionTest<sub>test</sub> | 66.9 | 70.5 |
| Video-MME<sub>wo/w subs</sub> | 63.3/69.0 | 65.1/71.6 |
| LVBench | | 45.3 |
| LongVideoBench | | 54.7 |
| MMBench-Video | 1.44 | 1.79 |
| TempCompass | | 71.7 |
| MLVU | | 70.2 |
| CharadesSTA/mIoU | 43.6|

Agent benchmark

| Benchmarks | Qwen2.5-VL-7B | |-------------------------|---------------| | ScreenSpot | 84.7 | | ScreenSpot Pro | 29.0 | | AITZ_EM | 81.9 | | Android Control High_EM | 60.1 | | Android Control Low_EM | 93.7 | | AndroidWorld_SR | 25.5 | | MobileMiniWob++_SR | 91.4 |

Requirements

The code of Qwen2.5-VL has been in the latest Hugging face transformers and we advise you to build from source with command:
code
pip install git+https://github.com/huggingface/transformers accelerate
or you might encounter the following error:
code
KeyError: 'qwen2_5_vl'

Quickstart

Below, we provide simple examples to show how to use Qwen2.5-VL with 🤖 ModelScope and 🤗 Transformers.

The code of Qwen2.5-VL has been in the latest Hugging face transformers and we advise you to build from source with command:

code
pip install git+https://github.com/huggingface/transformers accelerate

or you might encounter the following error:
code
KeyError: 'qwen2_5_vl'

We offer a toolkit to help you handle various types of visual input more conveniently, as if you were using an API. This includes base64, URLs, and interleaved images and videos. You can install it using the following command:

bash
# It's highly recommanded to use [decord] feature for faster video loading.
pip install qwen-vl-utils[decord]==0.0.8

If you are not using Linux, you might not be able to install decord from PyPI. In that case, you can use pip install qwen-vl-utils which will fall back to using torchvision for video processing. However, you can still install decord from source to get decord used when loading video.

Using 🤗 Transformers to Chat

Here we show a code snippet to show you how to use the chat model with transformers and qwen_vl_utils:

```python
from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from qwen_vl_utils import process_vision_info

default: Load the model on the available device(s)

model = Qwen2_5_VLForConditionalGeneration.from_pretrained( "Qwen/Qwen2.5-VL-7B-Instruct", torch_dtype="auto", device_map="auto" )

We recommend enabling flash_attention_2 for better acceleration and memory saving, especially in multi-image and video scenarios.

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(

"Qwen/Qwen2.5-VL-7B-Instruct",

torch_dtype=torch.bfloat16,

attn_implementation="flash_attention_2",

device_map="auto",

)

default processer

processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")

The default range for the number of visual tokens per image in the model is 4-16384.

You can set min_pixels and max_pixels according to your needs, such as a token range of 256-1280, to balance performance and cost.

min_pixels = 256*28*28

max_pixels = 1280*28*28

processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct", min_pixels=min_pixels, max_pixels=max_pixels)

messages = [
{
"role": "user",
"content": [
{
"type": "image",
"image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
},
{"type": "text", "text": "Describe this image."},
],
}
]

Preparation for inference

text = processor.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) image_inputs, vid
查看官方来源