uAI NEXUS MedVLM 1.0a 7B RL

---
language:

• en

• medical

• video understanding

• vision-language

• temporal action localization

• GRPO

• reinforcement learning

uAI-NEXUS-MedVLM-1.0a-7B-RL

> Accepted at CVPR 2026 🎉

Base Model: Qwen2.5-VL-7B-Instruct

uAI-NEXUS-MedVLM-1.0a-7B-RL is a medical-video understanding model fine-tuned from Qwen2.5-VL-7B-Instruct. It is the 7B-RL member of the uAI-NEXUS-MedVLM 1.0 family (variant a = Qwen2.5-VL base; variants b / c use Qwen3-VL-4B and Qwen3.5-4B respectively). Training uses a two-stage pipeline:

1. Supervised Fine-Tuning (SFT) on medical video QA data.
2. Group Relative Policy Optimization (GRPO) with task-specific rewards for temporal precision and clinical semantics.

It achieves state-of-the-art performance on medical video understanding across temporal action localization, spatiotemporal grounding, video summarization, region captioning, and surgical skill/CVS assessment.

• 📄 Paper: arXiv:2512.06581

• 🌐 Project Page: uii-ai.github.io/MedGRPO

• 💻 Code: github.com/UII-AI/MedGRPO-Code

• 🤗 Dataset: UII-AI/MedVidBench

• 📊 Leaderboard: UII-AI/MedVidBench-Leaderboard

Model Details

• Architecture: Qwen2.5-VL (7B parameters) — video + text

• Base Model: Qwen/Qwen2.5-VL-7B-Instruct

• Training: SFT → GRPO

• Domain: Medical and surgical video understanding

• License: Apache 2.0

Supported Tasks

The model handles 8 medical video understanding tasks (11 variants):

| Task Category | Tasks |
|---|---|
| Temporal Understanding | Temporal Action Localization (TAL), Spatiotemporal Grounding (STG), Next Action Prediction |
| Captioning | Dense Captioning (GPT / Gemini), Video Summary (GPT / Gemini), Region Caption (GPT / Gemini) |
| Assessment | Skill Assessment, CVS (Critical View of Safety) |

Training Data

Trained on 51,505 balanced video-instruction pairs (the MedVidBench Standard split), spanning 8 source datasets: AVOS, CholecT50, CholecTrack20, Cholec80-CVS, CoPESD, EgoSurgery, JIGSAWS, NurViD.

Stage 2 (GRPO) uses task-balanced subsets of the Standard split (detailed in the paper).

Training Details

Stage 1 — Supervised Fine-Tuning

• Objective: Learn medical video understanding from human-annotated QA pairs.

• Optimizer: AdamW with linear learning-rate schedule.

Stage 2 — Group Relative Policy Optimization (GRPO)

• Objective: Improve temporal precision and clinical semantic quality with RL.

• Reward functions:

• Framework: EasyR1 (built on verl by ByteDance).

Usage

Install

pip install transformers accelerate torch pillow qwen-vl-utils

Inference with Transformers

import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "UII-AI/uAI-NEXUS-MedVLM-1.0a-7B-RL",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("UII-AI/uAI-NEXUS-MedVLM-1.0a-7B-RL")
video_frames = ["frame_0001.jpg", "frame_0002.jpg", "frame_0003.jpg"]  # list of frame paths
messages = [{
    "role": "user",
    "content": [
        {"type": "video", "video": video_frames},
        {"type": "text", "text": "When does the surgeon grasp the gallbladder? Provide start and end times in seconds."},
    ],
}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text], images=image_inputs, videos=video_inputs,
    padding=True, return_tensors="pt",
).to(model.device)
with torch.no_grad():
    output_ids = model.generate(inputs, max_new_tokens=256)
    generated_ids = [out[len(inp):] for inp, out in zip(inputs.input_ids, output_ids)]
    response = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(response)
Example: "The surgeon grasps the gallbladder from 45.2 to 58.7 seconds."

git clone https://github.com/UII-AI/MedGRPO-Code
cd MedGRPO-Code
pip install -r requirements.txt
bash run_inference.sh

• Domain: Optimized for medical / surgical videos; may not generalize to other domains.

• Temporal Resolution: Best on videos sampled at 0.1–1.0 FPS.

• Language: Trained primarily on English medical terminology.

• Video Length**: Optimal for videos of a few minutes; longer videos rely on frame sub-sampling.

License

Released under the Apache 2.0 License.

Citation

If you use this model or the MedVidBench benchmark, please cite:

@inproceedings{su2026medgrpo,
  title     = {{MedGRPO}: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding},
  author    = {Su, Yuhao and Choudhuri, Anwesa and Gao, Zhongpai and Planche, Benjamin and
               Nguyen, Van Nguyen and Zheng, Meng and Shen, Yuhan and Innanje, Arun and
               Chen, Terrence and Elhamifar, Ehsan and Wu, Ziyan},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026}
}

Acknowledgments

• Base model: Qwen2.5-VL-7B-Instruct by Alibaba Cloud.

• Training framework: EasyR1 (built on verl).

Contact

Open an issue on the GitHub repository.