Name: OpenCaption 4B VL SFT v1.0 i1 GGUF
Author: mradermacher

Overview

OpenCaption 4B VL SFT v1.0 is a vision-language model optimized for high-accuracy image captioning and visual question answering. This specific GGUF quantization allows developers to deploy the model on consumer-grade hardware with significantly reduced VRAM overhead without sacrificing substantial reasoning capabilities. Unlike general-purpose LLMs with vision plugins, this SFT version is fine-tuned specifically for descriptive precision, making it ideal for automating alt-text generation, indexing visual assets, or building accessibility tools. It integrates seamlessly into local inference pipelines via llama.cpp, offering a lightweight alternative to massive proprietary vision models while maintaining strong spatial awareness and object recognition.

Highlights

GGUF quantization for efficient local CPU/GPU deployment
Fine-tuned for precise image captioning and VQA
Low memory footprint compared to larger VLMs
Apache-2.0 license for flexible commercial integration
Optimized for descriptive accuracy and visual reasoning

Usage

Install

# Install Hugging Face transformers
pip install transformers torch

SDK Usage

# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("mradermacher/OpenCaption-4B-VL-SFT-v1.0-i1-GGUF")
tokenizer = AutoTokenizer.from_pretrained("mradermacher/OpenCaption-4B-VL-SFT-v1.0-i1-GGUF")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance

pip install -U huggingface_hub

CLI Download

Download the full repository

huggingface-cli download mradermacher/OpenCaption-4B-VL-SFT-v1.0-i1-GGUF

Download a single file to a local folder (e.g. config.json into ./dir)

huggingface-cli download mradermacher/OpenCaption-4B-VL-SFT-v1.0-i1-GGUF config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('mradermacher/OpenCaption-4B-VL-SFT-v1.0-i1-GGUF')

Git Download

Make sure git-lfs is installed first

Git Download

git lfs install
git clone https://huggingface.co/mradermacher/OpenCaption-4B-VL-SFT-v1.0-i1-GGUF

To skip LFS large-file downloads, use:

Skip LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/mradermacher/OpenCaption-4B-VL-SFT-v1.0-i1-GGUF

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

pip install -U transformers torch

Load the model and run inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('mradermacher/OpenCaption-4B-VL-SFT-v1.0-i1-GGUF')
tokenizer = AutoTokenizer.from_pretrained('mradermacher/OpenCaption-4B-VL-SFT-v1.0-i1-GGUF')

Full Documentation

来源: HuggingFace

---
base_model: prithivMLmods/OpenCaption-4B-VL-SFT-v1.0
datasets:

prithivMLmods/OpenCaption-FineGrained

prithivMLmods/SuperFlickr-30K-LARGE-Remastered

prithivMLmods/OpenCaption-UHD

prithivMLmods/OpenCaption-Unified-10K

language:

en

library_name: transformers
license: apache-2.0
mradermacher:
readme_rev: 1
quantized_by: mradermacher
tags:

text-generation-inference

vision-language

multimodal

image-captioning

visual-question-answering

conditional-generation

vision

language-model

sft

fine-grained-captioning

computer-vision

vllm

---

About

weighted/imatrix quants of https://huggingface.co/prithivMLmods/OpenCaption-4B-VL-SFT-v1.0

*For a convenient overview and download list, visit our model page for this model.*

static quants are available at https://huggingface.co/mradermacher/OpenCaption-4B-VL-SFT-v1.0-GGUF

This is a vision model - mmproj files (if any) will be in the static repository.

Usage

If you are unsure how to use GGUF files, refer to one of TheBloke's
READMEs for
more details, including on how to concatenate multi-part files.

Provided Quants

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| GGUF | imatrix | 0.1 | imatrix file (for creating your own quants) |
| GGUF | i1-IQ1_S | 1.2 | for the desperate |
| GGUF | i1-IQ1_M | 1.2 | mostly desperate |
| GGUF | i1-IQ2_XXS | 1.3 | |
| GGUF | i1-IQ2_XS | 1.5 | |
| GGUF | i1-IQ2_S | 1.5 | |
| GGUF | i1-IQ2_M | 1.6 | |
| GGUF | i1-Q2_K_S | 1.7 | very low quality |
| GGUF | i1-Q2_K | 1.8 | IQ3_XXS probably better |
| GGUF | i1-IQ3_XXS | 1.8 | lower quality |
| GGUF | i1-IQ3_XS | 1.9 | |
| GGUF | i1-Q3_K_S | 2.0 | IQ3_XS probably better |
| GGUF | i1-IQ3_S | 2.0 | beats Q3_K* |
| GGUF | i1-IQ3_M | 2.1 | |
| GGUF | i1-Q3_K_M | 2.2 | IQ3_S probably better |
| GGUF | i1-Q3_K_L | 2.3 | IQ3_M probably better |
| GGUF | i1-IQ4_XS | 2.4 | |
| GGUF | i1-Q4_0 | 2.5 | fast, low quality |
| GGUF | i1-IQ4_NL | 2.5 | prefer IQ4_XS |
| GGUF | i1-Q4_K_S | 2.5 | optimal size/speed/quality |
| GGUF | i1-Q4_K_M | 2.6 | fast, recommended |
| GGUF | i1-Q4_1 | 2.7 | |
| GGUF | i1-Q5_K_S | 2.9 | |
| GGUF | i1-Q5_K_M | 3.0 | |
| GGUF | i1-Q6_K | 3.4 | practically like static Q6_K |

Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):

!image.png

And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.

Thanks

I thank my company, nethype GmbH, for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.