rtdetr v2 r18vd

提供商PekingU
分类object-detection
许可证apache-2.0
下载量372.9K
星标0

简介

RT-DETR v2 r18vd 是由北京大学团队开发的实时端到端目标检测模型。相比传统的 YOLO 系列，它最大的优势在于采用了 Transformer 架构且无需 NMS（非极大值抑制）后处理，有效解决了检测框冗余和推理延迟不稳定的痛点。r18vd 版本在保持轻量化的同时，优化了特征提取效率，非常适合部署在对实时性要求极高的工业缺陷检测、智能安防等边缘计算场景。对于习惯使用 PyTorch 的开发者来说，该模型上手难度较低，是追求高精度与低延迟平衡的理想替代方案。

核心亮点

  • 端到端检测无需 NMS，推理速度更稳定
  • 轻量化 r18 骨干网络，适配边缘端部署
  • Transformer 架构提供更强的全局感知能力
  • Apache-2.0 协议，商业化应用无压力

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("PekingU/rtdetr_v2_r18vd")
tokenizer = AutoTokenizer.from_pretrained("PekingU/rtdetr_v2_r18vd")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download PekingU/rtdetr_v2_r18vd

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download PekingU/rtdetr_v2_r18vd config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('PekingU/rtdetr_v2_r18vd')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/PekingU/rtdetr_v2_r18vd

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/PekingU/rtdetr_v2_r18vd

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('PekingU/rtdetr_v2_r18vd')
tokenizer = AutoTokenizer.from_pretrained('PekingU/rtdetr_v2_r18vd')

完整文档

来源: HuggingFace

---
library_name: transformers
license: apache-2.0
language:
- en
pipeline_tag: object-detection
tags:
- object-detection
- vision
datasets:
- coco
widget:
- src: >-
https://huggingface.co/datasets/mishig/sample_images/resolve/main/savanna.jpg
example_title: Savanna
- src: >-
https://huggingface.co/datasets/mishig/sample_images/resolve/main/football-match.jpg
example_title: Football Match
- src: >-
https://huggingface.co/datasets/mishig/sample_images/resolve/main/airport.jpg
example_title: Airport
---

RT-DETRv2

Overview

The RT-DETRv2 model was proposed in RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer by Wenyu Lv, Yian Zhao, Qinyao Chang, Kui Huang, Guanzhong Wang, Yi Liu. RT-DETRv2 refines RT-DETR by introducing selective multi-scale feature extraction, a discrete sampling operator for broader deployment compatibility, and improved training strategies like dynamic data augmentation and scale-adaptive hyperparameters.
These changes enhance flexibility and practicality while maintaining real-time performance.

This model was contributed by @jadechoghari with the help of @cyrilvallez and @qubvel-hf

This is

Performance

RT-DETRv2 consistently outperforms its predecessor across all model sizes while maintaining the same real-time speeds.

!rt-detr-v2-graph.png

How to use

python
import torch
import requests

from PIL import Image
from transformers import RTDetrV2ForObjectDetection, RTDetrImageProcessor


url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)


image_processor = RTDetrImageProcessor.from_pretrained("PekingU/rtdetr_v2_r18vd")
model = RTDetrV2ForObjectDetection.from_pretrained("PekingU/rtdetr_v2_r18vd")


inputs = image_processor(images=image, return_tensors="pt")


with torch.no_grad():
     outputs = model(inputs)


results = image_processor.post_process_object_detection(outputs, target_sizes=torch.tensor([(image.height, image.width)]), threshold=0.5)


for result in results:
     for score, label_id, box in zip(result["scores"], result["labels"], result["boxes"]):
         score, label = score.item(), label_id.item()
         box = [round(i, 2) for i in box.tolist()]
         print(f"{model.config.id2label[label]}: {score:.2f} {box}")

code
cat: 0.97 [341.14, 25.11, 639.98, 372.89]
cat: 0.96 [12.78, 56.35, 317.67, 471.34]
remote: 0.95 [39.96, 73.12, 175.65, 117.44]
sofa: 0.86 [-0.11, 2.97, 639.89, 473.62]
sofa: 0.82 [-0.12, 1.78, 639.87, 473.52]
remote: 0.79 [333.65, 76.38, 370.69, 187.48]

Training

RT-DETRv2 is trained on COCO (Lin et al. [2014]) train2017 and validated on COCO val2017 dataset. We report the standard AP metrics (averaged over uniformly sampled IoU thresholds ranging from 0.50 − 0.95 with a step size of 0.05), and APval50 commonly used in real scenarios.

Applications

RT-DETRv2 is ideal for real-time object detection in diverse applications such as autonomous driving, surveillance systems, robotics, and retail analytics**. Its enhanced flexibility and deployment-friendly design make it suitable for both edge devices and large-scale systems + ensures high accuracy and speed in dynamic, real-world environments.

查看官方来源