tiny doc qa vision encoder decoder

Providerfxmarty
Categorydocument-question-answering
Licensemit
Downloads5.5K
Stars0

Overview

The tiny-doc-qa-vision-encoder-decoder is a lightweight, specialized model designed for efficient document question answering (DocQA). Unlike general-purpose LLMs, this architecture leverages a vision encoder and a decoder to directly process document images, extracting textual information and spatial context to answer queries. It is particularly suited for developers building edge-computing applications or low-latency pipelines where deploying a massive multi-modal model is impractical. By focusing on the intersection of OCR and reading comprehension, it streamlines the workflow from raw image input to structured answer output, making it an ideal component for automated form processing or digital archive search tools.

Highlights

  • Optimized for low-latency document question answering tasks
  • Integrated vision encoder eliminates separate OCR preprocessing
  • Lightweight architecture suitable for edge device deployment
  • Permissive MIT license for flexible commercial integration

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("fxmarty/tiny-doc-qa-vision-encoder-decoder")
tokenizer = AutoTokenizer.from_pretrained("fxmarty/tiny-doc-qa-vision-encoder-decoder")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download fxmarty/tiny-doc-qa-vision-encoder-decoder

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download fxmarty/tiny-doc-qa-vision-encoder-decoder config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('fxmarty/tiny-doc-qa-vision-encoder-decoder')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/fxmarty/tiny-doc-qa-vision-encoder-decoder

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/fxmarty/tiny-doc-qa-vision-encoder-decoder

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('fxmarty/tiny-doc-qa-vision-encoder-decoder')
tokenizer = AutoTokenizer.from_pretrained('fxmarty/tiny-doc-qa-vision-encoder-decoder')

Full Documentation

来源: HuggingFace

---
license: mit
pipeline_tag: document-question-answering
---

For testing purposes only

View official source
Join our Telegram