Name: donut base finetuned docvqa
Author: Xenova

Overview

The Donut-based DocVQA model is an OCR-free transformer designed for document visual question answering. Unlike traditional pipelines that require a separate OCR engine to extract text before processing, this model reads the document image directly to extract information. This architectural shift reduces cumulative error rates and simplifies the deployment stack. It is particularly effective for structured documents like invoices, receipts, and forms where spatial layout is critical for context. Developers can integrate it into automated data extraction workflows or RAG pipelines requiring visual document understanding without the overhead of managing external OCR dependencies.

Highlights

OCR-free architecture eliminates external text extraction dependencies
Direct image-to-text mapping for faster document processing
Optimized for structured data extraction from visual documents
Apache-2.0 license ensures flexible commercial integration
Reduces pipeline complexity by merging vision and language

Usage

Install

# Install Hugging Face transformers
pip install transformers torch

SDK Usage

# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Xenova/donut-base-finetuned-docvqa")
tokenizer = AutoTokenizer.from_pretrained("Xenova/donut-base-finetuned-docvqa")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance

pip install -U huggingface_hub

CLI Download

Download the full repository

huggingface-cli download Xenova/donut-base-finetuned-docvqa

Download a single file to a local folder (e.g. config.json into ./dir)

huggingface-cli download Xenova/donut-base-finetuned-docvqa config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Xenova/donut-base-finetuned-docvqa')

Git Download

Make sure git-lfs is installed first

Git Download

git lfs install
git clone https://huggingface.co/Xenova/donut-base-finetuned-docvqa

To skip LFS large-file downloads, use:

Skip LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Xenova/donut-base-finetuned-docvqa

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

pip install -U transformers torch

Load the model and run inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Xenova/donut-base-finetuned-docvqa')
tokenizer = AutoTokenizer.from_pretrained('Xenova/donut-base-finetuned-docvqa')

Model Download

We recommend downloading the model via the ModelScope CLI or SDK.

Guidance：Before downloading, install ModelScope with:

Guidance

pip install modelscope

CLI Download

Download the full repository

modelscope download --model Xenova/donut-base-finetuned-docvqa

Download a single file to a local folder (e.g. README.md into ./dir)

modelscope download --model Xenova/donut-base-finetuned-docvqa README.md --local_dir ./dir

See the docs for more CLI options

SDK Download

# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Xenova/donut-base-finetuned-docvqa')

Git Download

Make sure git-lfs is installed first

Git Download

git lfs install
git clone https://www.modelscope.cn/Xenova/donut-base-finetuned-docvqa.git

To skip LFS large-file downloads, use:

Skip LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/Xenova/donut-base-finetuned-docvqa.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook Quickstart

Install the ModelScope library

pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

Load the model and run inference

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'Xenova/donut-base-finetuned-docvqa')

Full Documentation

来源: HuggingFace

---
base_model: naver-clova-ix/donut-base-finetuned-docvqa
library_name: transformers.js
pipeline_tag: document-question-answering
tags:

donut

image-to-text

vision

donut-swin

---

https://huggingface.co/naver-clova-ix/donut-base-finetuned-docvqa with ONNX weights to be compatible with Transformers.js.

Usage (Transformers.js)

If you haven't already, you can install the Transformers.js JavaScript library from NPM using:

bash

npm i @huggingface/transformers

Example: Answer questions about a document with Xenova/donut-base-finetuned-docvqa.

js

import { pipeline } from '@huggingface/transformers';
// Create a document question answering pipeline
const qa_pipeline = await pipeline('document-question-answering', 'Xenova/donut-base-finetuned-docvqa');
// Generate an answer for a given image and question
const image = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/invoice.png';
const question = 'What is the invoice number?';
const output = await qa_pipeline(image, question);
// [{ answer: 'us-001' }]

Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using 🤗 Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).