Get Started

YOLO11 vs RTDETRv2

Real-time detection increasingly involves a choice between convolutional neural networks (CNNs) and vision transformers (ViTs). This technical comparison covers two leading object detection architectures: YOLO11, a highly optimized CNN from Ultralytics, and RTDETRv2, an iteration of the Detection Transformer family from Baidu.

This guide compares their architectures, performance metrics, and deployment scenarios to help machine learning engineers choose between them. Both models reach high accuracy, but Ultralytics YOLO models typically offer a better balance of speed, ecosystem support, and ease of use in production.

YOLO11: The Benchmark for Real-World Versatility#

Introduced by Ultralytics, YOLO11 builds on years of YOLO research to deliver a model that is fast, accurate, and versatile. It natively handles object detection, instance segmentation, image classification, pose estimation, and oriented bounding box (OBB) detection.

Architecture and Strengths#

YOLO11 uses a refined CNN backbone and multi-scale feature pyramids, making it resource-efficient. It suits environments with strict hardware constraints, with a small memory footprint during both training and inference. Ultralytics Platform supports YOLO11 for data annotation, cloud training, and model monitoring without stitching together separate MLOps tools.

For edge deployment, YOLO11 offers low latency. Its lightweight design runs efficiently on devices ranging from Raspberry Pi boards to consumer mobile phones, making it a common choice for smart retail, manufacturing quality control, and automated traffic management.

RTDETRv2: Real-Time Transformers by Baidu#

RTDETRv2 (Real-Time Detection Transformer version 2) represents Baidu's effort to make transformer-based architectures viable for real-time tasks. It builds upon the original RT-DETR by incorporating a "bag-of-freebies" approach to improve baseline accuracy without inflating inference latency.

Learn more about RTDETRv2

Architecture and Strengths#

Unlike traditional CNNs, RTDETRv2 uses an encoder-decoder architecture with self-attention mechanisms, allowing it to capture global context across an image. This helps in crowded scenes where occlusions are frequent. RTDETRv2 removes the need for Non-Maximum Suppression (NMS) in post-processing, relying instead on Hungarian matching during training for one-to-one bipartite matching.

However, transformer models typically need more VRAM and CUDA memory. Training RTDETRv2 from scratch or fine-tuning it on custom datasets often requires high-end GPUs with large memory, which can be a barrier for smaller teams compared to the lighter training footprint of Ultralytics models.

Performance and Metrics Analysis#

Evaluated on the standard COCO dataset, the models show clear trade-offs between parameters, FLOPs, and accuracy (mAP).

Modelsize
(pixels)
mAPval50-95Speed
CPU ONNX
(ms)
Speed
T4 TensorRT10
(ms)
params
(M)
FLOPs
(B)
YOLO11n64039.556.11.52.66.5
YOLO11s64047.090.02.59.421.6
YOLO11m64051.5183.24.720.168.1
YOLO11l64053.4238.66.225.387.2
YOLO11x64054.7462.811.356.9195.3
RTDETRv2-s64048.1-5.032060
RTDETRv2-m64051.9-7.5136100
RTDETRv2-l64053.4-9.7642136
RTDETRv2-x64054.3-15.0376259

Unpacking the Results#

YOLO11 delivers more accuracy per parameter. YOLO11x reaches a higher mAPval50-95 (54.7) than RTDETRv2-x (54.3) while using fewer parameters (56.9M vs 76M) and fewer FLOPs (195.3B vs 259B).

YOLO11 is also faster on T4 TensorRT. YOLO11s runs in 2.5 ms versus 5.03 ms for RTDETRv2-s (47.0 vs 48.1 mAP), and YOLO11m is both more accurate (51.5 mAP) and faster (4.7 ms) than RTDETRv2-s. This makes YOLO11 a strong fit for real-time video analytics where frame processing time is the main bottleneck.

The Cost of Transformers

While RTDETRv2 achieves excellent accuracy through its attention layers, these mechanisms scale quadratically with image resolution, leading to higher VRAM consumption during both training and inference. YOLO11 avoids this cost with efficient convolutional blocks.

Training Ecosystem and Usability#

The core advantage of adopting an Ultralytics model lies in the surrounding ecosystem. Training RTDETRv2 often involves navigating complex research-grade repositories, adjusting intricate bipartite matching loss weights, and managing significant memory overhead.

Ultralytics focuses on developer experience. The unified Python API removes boilerplate code, integrates with tools like Weights & Biases for experiment tracking, and handles data augmentation automatically.

Here is how simple it is to train and export a model using the ultralytics package:

from ultralytics import YOLO

# Initialize YOLO11 model with pre-trained weights
model = YOLO("yolo11n.pt")

# Train the model efficiently on a local GPU or cloud instance
train_results = model.train(
    data="coco8.yaml",
    epochs=100,
    imgsz=640,
    device=0,  # Use CUDA GPU
)

# Export the trained model to ONNX for widespread deployment
export_path = model.export(format="onnx")

Once trained, exporting a YOLO11 model to formats like ONNX, OpenVINO, or CoreML requires a single command, so the same model can run across diverse hardware backends.

Multi-Task Capabilities

While RTDETRv2 focuses exclusively on bounding box detection, YOLO11 also supports instance segmentation, image classification, pose estimation, and OBB detection, so you can consolidate multiple vision tasks into a single model family.

Use Cases and Recommendations#

Choosing between YOLO11 and RT-DETR depends on your specific project requirements, deployment constraints, and ecosystem preferences.

When to Choose YOLO11#

YOLO11 is a strong choice for:

  • Production Edge Deployment: Commercial applications on devices like Raspberry Pi or NVIDIA Jetson where reliability and active maintenance are paramount.
  • Multi-Task Vision Applications: Projects requiring detection, segmentation, classification, pose estimation, and OBB within a single unified framework.
  • Rapid Prototyping and Deployment: Teams that need to move quickly from data collection to production using the streamlined Ultralytics Python API.

When to Choose RT-DETR#

RT-DETR is recommended for:

  • Transformer-Based Detection Research: Projects exploring attention mechanisms and transformer architectures for end-to-end object detection without NMS.
  • High-Accuracy Scenarios with Flexible Latency: Applications where detection accuracy is the top priority and slightly higher inference latency is acceptable.
  • Large Object Detection: Scenes with primarily medium-to-large objects where the global attention mechanism of transformers provides a natural advantage.

When to Choose Ultralytics (YOLO26)#

For most new projects, Ultralytics YOLO26 offers the best combination of performance and developer experience:

  • NMS-Free Edge Deployment: Applications requiring consistent, low-latency inference without the complexity of Non-Maximum Suppression post-processing.
  • CPU-Only Environments: Devices without dedicated GPU acceleration, where YOLO26's up to 43% faster CPU inference (YOLO26n vs YOLO11n) provides a decisive advantage.
  • Small Object Detection: Challenging scenarios like aerial drone imagery or IoT sensor analysis where ProgLoss and STAL improve accuracy on small objects.

Looking Ahead: The Power of YOLO26#

YOLO11 remains a solid production choice, but new projects should start with YOLO26. Released in January 2026, YOLO26 narrows the architectural gap with transformers through an optional end-to-end NMS-free head (nms=False, introduced to YOLO models by YOLOv10), which removes NMS post-processing latency and simplifies deployment logic.

YOLO26 also introduces several other changes:

  • MuSGD Optimizer: Inspired by the large language model (LLM) training techniques of Moonshot AI's Kimi K2, this hybrid of SGD and Muon provides more stable training and faster convergence.
  • DFL Removal: Removing Distribution Focal Loss simplifies export and improves compatibility with low-power edge devices.
  • ProgLoss + STAL: Progressive Loss and Small-Target-Aware Label Assignment (STAL) improve small-object recognition, a key requirement for drone surveillance, agricultural monitoring, and IoT edge sensors.
  • Up to 43% Faster CPU Inference: For deployments without dedicated GPUs, YOLO26n runs up to 43% faster than YOLO11n on CPU.

For a wider range of architectures, see YOLOv8, the widely adopted YOLOv5, and specialized models like YOLO-World for open-vocabulary detection. Whether you choose the proven stability of YOLO11 or the newer YOLO26, the Ultralytics ecosystem provides the tools to take your computer vision projects to production.

Comments

Ask a question, share your results, or help someone else.