tiny doc qa vision encoder decoder
Overview
The Tiny Doc QA Vision Encoder-Decoder is a lightweight model designed specifically for document-based visual question answering. Unlike general-purpose LLMs, this architecture combines a vision encoder to parse document layouts with a decoder to generate precise answers from the extracted visual context. It is ideal for developers building edge-deployed OCR pipelines or automated data extraction tools where low latency and a small memory footprint are critical. By focusing on the intersection of spatial layout and text, it avoids the overhead of massive parameter counts while maintaining high accuracy for structured document retrieval. Integration is straightforward for those already using the Optimum-Intel framework, offering a streamlined path for deploying document intelligence on CPU-constrained environments.
Highlights
- Optimized for low-latency document visual question answering
- Lightweight encoder-decoder architecture for edge deployment
- Efficient spatial layout parsing and text extraction
- Seamless integration via the Optimum-Intel framework
- Permissive MIT license for flexible commercial use
Usage
Install
# Install Hugging Face transformers
pip install transformers torchSDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("optimum-intel-internal-testing/tiny-doc-qa-vision-encoder-decoder")
tokenizer = AutoTokenizer.from_pretrained("optimum-intel-internal-testing/tiny-doc-qa-vision-encoder-decoder")Hugging Face Download
We recommend downloading the model via the Hugging Face CLI or Hub SDK.
Guidance：Before downloading, install huggingface_hub with:
Guidance
pip install -U huggingface_hubCLI Download
Download the full repository
Download the full repository
huggingface-cli download optimum-intel-internal-testing/tiny-doc-qa-vision-encoder-decoderDownload a single file to a local folder (e.g. config.json into ./dir)
Download a single file to a local folder (e.g. config.json into ./dir)
huggingface-cli download optimum-intel-internal-testing/tiny-doc-qa-vision-encoder-decoder config.json --local-dir ./dirSee the official docs for more CLI options
SDK Download
SDK Download
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('optimum-intel-internal-testing/tiny-doc-qa-vision-encoder-decoder')Git Download
Make sure git-lfs is installed first
Git Download
git lfs install
git clone https://huggingface.co/optimum-intel-internal-testing/tiny-doc-qa-vision-encoder-decoderTo skip LFS large-file downloads, use:
Skip LFS
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/optimum-intel-internal-testing/tiny-doc-qa-vision-encoder-decoderModel files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.
PyTorch / Transformers Usage
Install Transformers
Install Transformers
pip install -U transformers torchLoad the model and run inference
Load the model and run inference
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained('optimum-intel-internal-testing/tiny-doc-qa-vision-encoder-decoder')
tokenizer = AutoTokenizer.from_pretrained('optimum-intel-internal-testing/tiny-doc-qa-vision-encoder-decoder')Full Documentation
来源: HuggingFace
---
license: mit
pipeline_tag: document-question-answering
---
For testing purposes only