Ultralytics YOLO27:
Get Started

Ultralytics Benchmark#

COCO is not representative of many real-world computer vision tasks. Ultralytics Benchmark is a benchmark dataset for measuring model generalizability across 37 curated object detection datasets with high-quality bounding box labels. Its domains include medical image analysis, remote sensing, traffic, documents, user interfaces, retail, machine vision, agriculture, sports, wildlife, underwater imaging, and waste detection. Its sensor types include RGB, fisheye, thermal infrared, X-ray, MRI, ultrasound, microscopy, and synthetic aperture radar (SAR).

The curated datasets are hosted on Ultralytics Platform.

Candidate datasets were checked for exact and near-duplicate images across the training and validation splits. Flagged pairs were reviewed, and confirmed split leaks were removed or the affected dataset was replaced.

Datasets#

DomainSensor / imageryDatasetClasses
Aerial/RSRGB satellite imageryAirbus Oil Storage Detection1: oil-storage-tank
Document/UIRGB and scanned pagesAnnoPage document elements25: chemical formula/equation, symbol/logo, photograph, drawing, graph, map, mathematical expression, musical notation, image, decor, inscription, floor plan, barcode/QR code, stamp, advertisement, handwritten note, diagram, table, vignette, and frieze
SportsRGB cameraBasketball Hoop Detection2: basketball, basketball net
Driving/TrafficRGB road cameraCEYMO11: bus lane, cycle lane, diamond, junction box, left arrow, pedestrian crossing, right arrow, straight arrow, slow, straight-left arrow, straight-right arrow
MedicalMRICheng Brain Tumor MRI3: meningioma, glioma, pituitary
MedicalColor fundus cameraDDR Lesion4: hard exudates, hemorrhages, microaneurysms, soft exudates
IndustrialLinear-scan industrial cameraDeepPCB6: open, short, mousebite, spur, copper, pin-hole
MedicalDental X-rayDENTEX4: Impacted, Caries, Periapical Lesion, Deep Caries
IndustrialRGB industrial cameraDSPCBSD9: short, spur, spurious copper, open, mouse bite, hole breakout, conductor scratch, conductor foreign object, base material foreign object
OtherRGB egocentric cameraEgoHands4: own_left, own_right, other_left, other_right
OtherRGB and infrared trail cameraENA2422 wildlife, person, domestic-animal, and vehicle classes
Driving/TrafficRGB fisheye cameraFisheye8K5: Bus, Bike, Car, Pedestrian, Truck
Document/UIDigital and scanned plansFloorPlanCAD30 architectural opening, furniture, appliance, bathroom fixture, stair, elevator, and escalator classes
MedicalX-rayGRAZPEDWRI-DX9: boneanomaly, bonelesion, foreignbody, fracture, metal, periostealreaction, pronatorsign, softtissue, text
Document/UIDigital screenshotsGroundCUA UI9: Button, Information Display, Input Elements, Menu, Navigation, Others, Sidebar, Visual Elements, Unknown
AgricultureRGB field cameraGWHD 20211: wheat_head
Aerial/RSThermal infrared UAVHIT-UAV5: person, car, bicycle, other vehicle, ignored region
MedicalSecurity X-rayIED X-ray11 explosive, battery, laptop, pager, mobile phone, walkie-talkie, and modified-device classes
MedicalRGB endoscopeKvasir-SEG Polyp1: polyp
Driving/TrafficThermal infraredLLVIP1: person
Aerial/RS8-band multispectral imageryMODA Multispectral8: car, van, truck, bus, tricycle, bike, awning-bike, pedestrian
AgricultureRGB cameraPlantDoc29 crop leaf and plant-disease classes
SportsRGB video framesRacketVision Tennis Ball1: tennis ball
Driving/TrafficRGB road cameraRDD20225: longitudinal crack, transverse crack, alligator crack, other corruption, pothole
RetailRGB checkout cameraRPC Retail Checkout200 checkout product classes covering packaged food, drinks, milk, canned food, chocolate, gum, candy, seasonings, hygiene products, tissues, and stationery
OtherRGB underwater cameraRUOD Underwater10: holothurian, echinus, scallop, starfish, fish, corals, diver, cuttlefish, turtle, jellyfish
IndustrialRGB cameraSafe Human 1717 person, body-part, tool, and personal protective equipment classes
Aerial/RSRGB UAV cameraSeaDronesSee6: ignored, swimmer, boat, jetski, life_saving_appliances, buoy
RetailRGB retail cameraSKU-110K1: shelf product
SportsRGB broadcast cameraSoccerDB3: player, ball, goal
Aerial/RSSynthetic aperture radar (SAR)SSDD SAR Ship1: ship
OtherRGB cameraTACO Litter28 litter material and object classes
MedicalChest X-rayTBX11K2: ActiveTuberculosis, ObsoletePulmonaryTuberculosis
MedicalUltrasoundTN5000 Thyroid Ultrasound2: benign, malignant
MedicalLight microscopyTXL-PBC Blood Cells3: white blood cells, red blood cells, platelets
OtherRGB cameraUEC School Lunch Food21 school-lunch food and drink classes
Aerial/RSRGB UAV cameraVisDrone10: pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus, motor

Evaluation#

  • Sample up to 1,000 images from each training split, keep the full validation split, and exclude the test split with fraction=[1000, 1.0, 0.0].
  • Fine-tune the same pretrained checkpoint independently on each dataset for 100 epochs with patience 100, seed 0, batch 16, and four workers. Images use 640 pixels. Seed 0 is not fully deterministic, so YOLO27m and YOLO27l report the mean of 3 such runs.
  • Enable class remapping, channels-last, and prediction JSON output, with plots disabled.
  • Evaluate each dataset's best checkpoint and reuse its saved max_det, which accounts for its largest object count.
  • For size-specific mAP50-95, multiply ground-truth and prediction box areas by (640 / max(image_width, image_height)) ** 2 before applying the COCO Small, Medium, and Large bins.
  • Give every dataset equal weight. Each size metric includes only datasets with at least 10 objects in that bin.

Macro mAP is sum(dataset validation mAP50-95) / 37. Metrics are calculated from each dataset's saved predictions.

Benchmark results#

Metrics are macro averages across the 37 validation splits, reported as percentages.

SizeModelmAP50-95PrecisionRecallmAP50-95-SmAP50-95-MmAP50-95-L
L/XYOLO27l55.5078.8374.9038.1755.7563.36
L/XYOLO26x49.8576.7468.6134.4450.5451.69
L/XYOLO26l49.7275.0569.3134.4950.1651.72
L/XYOLO11l48.5374.2268.3331.9150.7150.77
L/XYOLOv5l48.0374.1167.7831.7349.7949.87
L/XYOLOv8l48.0273.9568.8032.0549.0449.56
L/XYOLOv8x47.6775.6666.4631.1249.6049.69
L/XYOLOv5x47.3574.6967.9631.2749.3949.76
L/XYOLO11x47.2473.6967.0831.2248.5749.24
MYOLO27m53.3875.8572.9535.9653.4061.40
MYOLO26m49.4276.2368.3132.9150.0251.32
MYOLOv5m48.1974.2168.0731.6849.6751.93
MYOLO11m48.1573.7268.5632.1449.7650.75
MYOLOv8m47.9973.8668.1530.7349.6151.97
SYOLO27s51.2376.9670.7434.2051.4754.74
SYOLO11s47.7874.7267.6629.5349.6950.69
SYOLO26s47.4774.3566.7331.5647.7250.58
SYOLOv8s47.3573.9767.5430.9048.4750.15
SYOLOv5s47.3073.9767.0230.4749.0750.10
NYOLO27n49.2574.8668.4931.9349.3053.20
NYOLO11n46.1972.8365.6928.3547.5150.66
NYOLOv8n45.8972.2965.5628.5647.5549.22
NYOLOv5n45.0971.6964.9928.6146.7948.14
NYOLO26n44.0870.0963.2028.6944.5247.96

Benchmarking#

Use the main branch of Ultralytics with ULTRALYTICS_API_KEY set for dataset access:

from ultralytics import YOLO

datasets = [
    "ul://ultralytics/datasets/airbus-oil-storage-detection",
    "ul://ultralytics/datasets/annopage-doc-elements",
    "ul://ultralytics/datasets/basketball-hoop-detection",
    "ul://ultralytics/datasets/ceymo",
    "ul://ultralytics/datasets/cheng-brain-tumor-mri",
    "ul://ultralytics/datasets/ddr-lesion",
    "ul://ultralytics/datasets/deeppcb",
    "ul://ultralytics/datasets/dentex",
    "ul://ultralytics/datasets/dspcbsd-pcb-surface-defect",
    "ul://ultralytics/datasets/egohands-egocentric",
    "ul://ultralytics/datasets/ena24",
    "ul://ultralytics/datasets/fisheye8k-traffic",
    "ul://ultralytics/datasets/floorplancad-2",
    "ul://ultralytics/datasets/grazpedwri-dx",
    "ul://ultralytics/datasets/groundcua-ui-yolo",
    "ul://ultralytics/datasets/global-wheat-head-detection-2021",
    "ul://ultralytics/datasets/hit-uav",
    "ul://ultralytics/datasets/iedxray",
    "ul://ultralytics/datasets/kvasir-seg-polyp",
    "ul://ultralytics/datasets/llvip",
    "ul://ultralytics/datasets/moda-multispectral-obb",
    "ul://ultralytics/datasets/plantdoc",
    "ul://ultralytics/datasets/racketvision-tennis-ball-yolo",
    "ul://ultralytics/datasets/rdd2022",
    "ul://ultralytics/datasets/rpc-retail-checkout",
    "ul://ultralytics/datasets/ruod-underwater",
    "ul://ultralytics/datasets/safe-human-17",
    "ul://ultralytics/datasets/seadronessee-2",
    "ul://ultralytics/datasets/sku110k-2",
    "ul://ultralytics/datasets/soccerdb",
    "ul://ultralytics/datasets/ssdd-sar-ship",
    "ul://ultralytics/datasets/taco-detect",
    "ul://ultralytics/datasets/tbx11k-tb-only",
    "ul://ultralytics/datasets/tn5000-thyroid-us",
    "ul://ultralytics/datasets/txl-pbc-blood-cells",
    "ul://ultralytics/datasets/uec-school-lunch-food-2",
    "ul://ultralytics/datasets/visdrone",
]

results = YOLO("yolo27n.pt").train(
    data=datasets,
    epochs=100,
    patience=100,
    imgsz=640,
    batch=16,
    fraction=[1000, 1.0, 0.0],
    seed=0,
    workers=4,
    cls_remap=True,
    channels_last=True,
    save_json=True,
    plots=False,
)
# The main branch reports native-area size bins. This benchmark separately rescales GT and prediction areas
# by (640 / max(image_width, image_height)) ** 2 before scoring size-specific AP.

Citation#

If you use Ultralytics Benchmark in your work, please cite the Ultralytics YOLO26 paper:

@misc{jocher2026ultralyticsyolo26unifiedrealtime,
  title = {Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models},
  author = {Glenn Jocher and Jing Qiu and Mengyu Liu and Shuai Lyu and Fatih Cagatay Akyon and Muhammet Esat Kalfaoglu},
  year = {2026},
  eprint = {2606.03748},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  doi = {10.48550/arXiv.2606.03748},
  url = {https://arxiv.org/abs/2606.03748},
}
Contributors

Comments