rtdetr v2 r18vd

ProviderPekingU
Categoryobject-detection
Licenseapache-2.0
Downloads372.9K
Stars0

Overview

RT-DETR v2 (R18VD) is a high-performance real-time object detector that bridges the gap between the speed of YOLO-style architectures and the accuracy of Transformer-based models. By utilizing a ResNet-18 backbone with a refined hybrid encoder, it eliminates the need for Non-Maximum Suppression (NMS), significantly reducing post-processing latency and simplifying the deployment pipeline. For developers, this means more predictable inference times and easier integration into edge computing environments. It is particularly effective for industrial automation, robotics, and real-time video analytics where both low latency and high precision are critical requirements. Compared to its predecessors, v2 offers improved convergence and better feature extraction, making it a robust choice for custom dataset fine-tuning via the Apache-2.0 license.

Highlights

  • Real-time detection without NMS post-processing overhead
  • Optimized ResNet-18 backbone for efficient edge deployment
  • Transformer-based architecture for superior global context awareness
  • Permissive Apache-2.0 license for commercial integration
  • Balanced trade-off between inference speed and mAP

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("PekingU/rtdetr_v2_r18vd")
tokenizer = AutoTokenizer.from_pretrained("PekingU/rtdetr_v2_r18vd")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download PekingU/rtdetr_v2_r18vd

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download PekingU/rtdetr_v2_r18vd config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('PekingU/rtdetr_v2_r18vd')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/PekingU/rtdetr_v2_r18vd

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/PekingU/rtdetr_v2_r18vd

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('PekingU/rtdetr_v2_r18vd')
tokenizer = AutoTokenizer.from_pretrained('PekingU/rtdetr_v2_r18vd')

Full Documentation

来源: HuggingFace

---
library_name: transformers
license: apache-2.0
language:
- en
pipeline_tag: object-detection
tags:
- object-detection
- vision
datasets:
- coco
widget:
- src: >-
https://huggingface.co/datasets/mishig/sample_images/resolve/main/savanna.jpg
example_title: Savanna
- src: >-
https://huggingface.co/datasets/mishig/sample_images/resolve/main/football-match.jpg
example_title: Football Match
- src: >-
https://huggingface.co/datasets/mishig/sample_images/resolve/main/airport.jpg
example_title: Airport
---

RT-DETRv2

Overview

The RT-DETRv2 model was proposed in RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer by Wenyu Lv, Yian Zhao, Qinyao Chang, Kui Huang, Guanzhong Wang, Yi Liu. RT-DETRv2 refines RT-DETR by introducing selective multi-scale feature extraction, a discrete sampling operator for broader deployment compatibility, and improved training strategies like dynamic data augmentation and scale-adaptive hyperparameters.
These changes enhance flexibility and practicality while maintaining real-time performance.

This model was contributed by @jadechoghari with the help of @cyrilvallez and @qubvel-hf

This is

Performance

RT-DETRv2 consistently outperforms its predecessor across all model sizes while maintaining the same real-time speeds.

!rt-detr-v2-graph.png

How to use

python
import torch
import requests

from PIL import Image
from transformers import RTDetrV2ForObjectDetection, RTDetrImageProcessor


url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)


image_processor = RTDetrImageProcessor.from_pretrained("PekingU/rtdetr_v2_r18vd")
model = RTDetrV2ForObjectDetection.from_pretrained("PekingU/rtdetr_v2_r18vd")


inputs = image_processor(images=image, return_tensors="pt")


with torch.no_grad():
     outputs = model(inputs)


results = image_processor.post_process_object_detection(outputs, target_sizes=torch.tensor([(image.height, image.width)]), threshold=0.5)


for result in results:
     for score, label_id, box in zip(result["scores"], result["labels"], result["boxes"]):
         score, label = score.item(), label_id.item()
         box = [round(i, 2) for i in box.tolist()]
         print(f"{model.config.id2label[label]}: {score:.2f} {box}")

code
cat: 0.97 [341.14, 25.11, 639.98, 372.89]
cat: 0.96 [12.78, 56.35, 317.67, 471.34]
remote: 0.95 [39.96, 73.12, 175.65, 117.44]
sofa: 0.86 [-0.11, 2.97, 639.89, 473.62]
sofa: 0.82 [-0.12, 1.78, 639.87, 473.52]
remote: 0.79 [333.65, 76.38, 370.69, 187.48]

Training

RT-DETRv2 is trained on COCO (Lin et al. [2014]) train2017 and validated on COCO val2017 dataset. We report the standard AP metrics (averaged over uniformly sampled IoU thresholds ranging from 0.50 − 0.95 with a step size of 0.05), and APval50 commonly used in real scenarios.

Applications

RT-DETRv2 is ideal for real-time object detection in diverse applications such as autonomous driving, surveillance systems, robotics, and retail analytics**. Its enhanced flexibility and deployment-friendly design make it suitable for both edge devices and large-scale systems + ensures high accuracy and speed in dynamic, real-world environments.

View official source
Join our Telegram