Get Started

RTDETRv2 vs YOLO11

New computer vision architectures keep improving what is possible on edge devices and cloud servers. RTDETRv2 and YOLO11 are two widely used options for real-time object detection. Both perform well, but they represent different architectural approaches: a transformer-based detector versus a highly optimized convolutional neural network (CNN).

This comparison covers the architectures, performance metrics, training methodologies, and ideal use cases of both models to help you choose the right one for your next application.

RTDETRv2: The Transformer-Based Challenger#

Introduced as an evolution of the original Real-Time Detection Transformer, RTDETRv2 applies attention mechanisms to multi-scale features from a CNN backbone. Its hybrid encoder and transformer decoder capture global image context, which helps when detecting heavily overlapping objects in complex scenes.

Model Details:

Architectural Strengths and Weaknesses#

RTDETRv2's primary innovation is its end-to-end NMS-free architecture. By removing Non-Maximum Suppression (NMS), it simplifies the post-processing pipeline. Furthermore, its multi-scale feature extraction capabilities have been improved over the original RT-DETR model, allowing it to better identify objects of varying sizes.

However, because it relies on transformers, RTDETRv2 typically has higher memory requirements during training. Transformers are generally slower to converge and require more CUDA memory than traditional CNNs, making them less accessible for researchers operating on consumer-grade hardware or deploying to constrained edge AI environments.

Learn more about RTDETRv2

Ultralytics YOLO11: CNN Efficiency#

Building on earlier YOLO research, Ultralytics YOLO11 refines the CNN architecture for a better speed-accuracy trade-off while keeping the flexible, developer-friendly Ultralytics ecosystem.

Model Details:

The Ultralytics Advantage#

YOLO11's main strength is its Performance Balance. Its trade-off between speed and accuracy suits a wide range of deployment scenarios, from large cloud computing clusters to lightweight mobile devices.

Ultralytics YOLO models also use less memory during training and inference. Unlike transformer models, which can quickly exhaust VRAM, YOLO11 allows larger batch sizes on standard GPUs. YOLO11 also offers Versatility beyond object detection, with native support for Instance Segmentation, Image Classification, Pose Estimation, and Oriented Bounding Boxes (OBB).

Performance and Metrics Comparison#

The table below compares COCO accuracy and speed. RTDETRv2 is slightly more accurate at the small and medium scales, while YOLO11 offers five model sizes, including a nano variant, and lower TensorRT latency at every comparable size.

Modelsize
(pixels)
mAPval50-95Speed
CPU ONNX
(ms)
Speed
T4 TensorRT10
(ms)
params
(M)
FLOPs
(B)
RTDETRv2-s64048.1-5.032060
RTDETRv2-m64051.9-7.5136100
RTDETRv2-l64053.4-9.7642136
RTDETRv2-x64054.3-15.0376259
YOLO11n64039.556.11.52.66.5
YOLO11s64047.090.02.59.421.6
YOLO11m64051.5183.24.720.168.1
YOLO11l64053.4238.66.225.387.2
YOLO11x64054.7462.811.356.9195.3

YOLO11x reaches 54.7% mAPval, 0.4 points above RTDETRv2-x, with fewer FLOPs (195.3B vs 259B) and faster TensorRT inference (11.3 ms vs 15.03 ms). YOLO11l matches RTDETRv2-l at 53.4% while running in 6.2 ms instead of 9.76 ms, and RTDETRv2-s and -m lead YOLO11s and -m by 1.1 and 0.4 points at 1.6-2x the latency. The nano and small YOLO11 variants are lightweight options for constrained devices like the Raspberry Pi.

Ecosystem, Ease of Use, and Training#

Ultralytics models are designed for a straightforward user experience. The ultralytics Python package provides a unified API that handles data augmentation, distributed training, and model export. RTDETRv2's research repository requires more boilerplate and configuration, while Ultralytics provides a complete pipeline from data to deployment.

The Ultralytics package also runs RT-DETR models alongside YOLO models, so both can use the Well-Maintained Ecosystem of Ultralytics, including integrations with Weights & Biases and Comet for experiment tracking.

from ultralytics import RTDETR, YOLO

# Load an RTDETR model through the Ultralytics API
model_rtdetr = RTDETR("rtdetr-l.pt")

# Load a highly optimized YOLO11 model
model_yolo = YOLO("yolo11n.pt")

# Train YOLO11 with highly efficient memory usage
results = model_yolo.train(data="coco8.yaml", epochs=100, imgsz=640)

# Export the trained YOLO model to ONNX format
model_yolo.export(format="onnx")
Streamline Your Workflow

Training efficiency matters in machine learning. Ultralytics models start from pretrained weights, so transfer learning on custom data converges quickly. To manage datasets, training runs, and deployment endpoints without writing code, use Ultralytics Platform for an integrated MLOps workflow.

Real-World Applications#

Choosing between these architectures often comes down to the specific deployment constraints of your project.

Where RTDETRv2 Excels: RTDETRv2's Transformer backbone is highly effective in scenarios with dense, heavily occluded objects where global context is required. It is often used in academic research and in applications where computational budget matters less than modeling relationships between objects.

Where YOLO11 Excels: YOLO11 is well suited to practical, real-world deployment. Its small memory footprint and fast inference make it a good fit for:

  • Smart Manufacturing: Running real-time defect detection on production lines using industrial PCs.
  • Agriculture: Deploying on drones for real-time crop health monitoring and automated harvesting robotics.
  • Retail Analytics: Processing multiple camera streams concurrently for queue management and inventory tracking without requiring large server clusters.

Use Cases and Recommendations#

Choosing between RT-DETR and YOLO11 depends on your specific project requirements, deployment constraints, and ecosystem preferences.

When to Choose RT-DETR#

RT-DETR is a strong choice for:

  • Transformer-Based Detection Research: Projects exploring attention mechanisms and transformer architectures for end-to-end object detection without NMS.
  • High-Accuracy Scenarios with Flexible Latency: Applications where detection accuracy is the top priority and slightly higher inference latency is acceptable.
  • Large Object Detection: Scenes with primarily medium-to-large objects where the global attention mechanism of transformers provides a natural advantage.

When to Choose YOLO11#

YOLO11 is recommended for:

  • Production Edge Deployment: Commercial applications on devices like Raspberry Pi or NVIDIA Jetson where reliability and active maintenance are paramount.
  • Multi-Task Vision Applications: Projects requiring detection, segmentation, classification, pose estimation, and OBB within a single unified framework.
  • Rapid Prototyping and Deployment: Teams that need to move quickly from data collection to production using the streamlined Ultralytics Python API.

When to Choose Ultralytics (YOLO26)#

For most new projects, Ultralytics YOLO26 offers the best combination of performance and developer experience:

  • NMS-Free Edge Deployment: Applications requiring consistent, low-latency inference without the complexity of Non-Maximum Suppression post-processing.
  • CPU-Only Environments: Devices without dedicated GPU acceleration, where YOLO26n's up to 43% faster CPU inference than YOLO11n provides a decisive advantage.
  • Small Object Detection: Challenging scenarios like aerial drone imagery or IoT sensor analysis where ProgLoss and STAL improve accuracy on small objects.

Looking Forward: The Arrival of YOLO26#

If you are beginning a new project, also consider Ultralytics YOLO26, released in January 2026 and the recommended Ultralytics model. It combines ideas from both approaches: an optional End-to-End NMS-Free head (nms=False, introduced to YOLO models by YOLOv10) removes NMS post-processing like RTDETRv2 while keeping the speed of a CNN.

YOLO26 uses the MuSGD Optimizer, inspired by LLM training techniques, for stable and fast convergence, and YOLO26n delivers up to 43% Faster CPU Inference than YOLO11n, helped by the removal of Distribution Focal Loss (DFL). Progressive Loss and Small-Target-Aware Label Assignment (ProgLoss + STAL) improve small-object recognition, making YOLO26 the recommended choice for new computer vision pipelines.

Whether you choose YOLO11 for its proven versatility, RTDETRv2 for its attention mechanisms, or YOLO26 for edge performance, the Ultralytics documentation covers training and deployment for YOLO11, YOLO26, and the original RT-DETR.

Comments1

Ask a question, share your results, or help someone else.

  1. UltralyticsAssistantGitHubUltralytics

    👋 Hello, thank you for starting this discussion about comparing RTDETRv2 and YOLO11 🚀! We’re thrilled to see your interest in exploring the strengths and ideal use cases of these models.

    For those looking to dive deeper, the Comparison Guide is a great resource to start with. It highlights key differences and performance metrics, helping you make informed decisions based on your project needs.

    If you have specific questions about YOLO11 performance or implementation, feel free to provide more details so we can assist better. For custom training or real-world applications, including dataset examples or logs would be immensely helpful!

    💡 Pro Tip: Keep your ultralytics package up to date to leverage the latest features and fixes. Simply run:

    pip install -U ultralytics

    For real-time discussions, join our Discord 🎧. Alternatively, connect with our community on Discourse or our Subreddit to share insights and experiences.

    An Ultralytics engineer will review and assist with your discussion soon. Stay tuned! 😊