Depth Estimation Datasets Overview#

Monocular depth estimation assigns a depth value in meters to every pixel in an image. Depth targets use either scaled 16-bit grayscale PNGs or floating-point NPY arrays in meters.

This guide explains the dataset format used by Ultralytics YOLO depth estimation models and lists the built-in dataset configurations available for training and validation.

Supported Dataset Format#

Depth map format#

Each training sample consists of one RGB image and one paired depth file. PNG values are divided by the optional dataset-level depth_scale to produce meters. The default is 1000, so a value of 1500 represents 1.5 meters. Code 0 means invalid; PNGs need no embedded metadata.

• Depth files use .png (preferred) or .npy. PNG maps must be 2D uint16 grayscale images. NPY arrays must be 2D and floating-point, with values in meters.
• Set depth_scale only when PNGs do not use the default millimeter convention. For example, KITTI uses 256 and Virtual KITTI 2 uses 100.
• Each depth file should have the same stem as its matching image file (e.g., scene_001.png pairs with scene_001.jpg).
• Depth maps may be smaller than their RGB images as long as the aspect ratio matches; training resizes them in memory.
• The dataset loader finds depth files by replacing the images directory component with depth, preferring .png and falling back to .npy.
• Pixels with depth ≤ 0 are treated as invalid and excluded from loss and metric computation.

Because code 0 is reserved for invalid pixels, a uint16 PNG provides 65,535 positive depth values. The scale controls both precision and range:

These are storage limits, not recommended training caps. A dataset can set a smaller max_depth independently for loss calibration or evaluation.

The standard layout keeps images and depth maps in parallel folders:

dataset/
├── images/
│   ├── train/
│   └── val/
└── depth/
    ├── train/
    └── val/

For example, an image at images/train/scene_001.jpg is paired with a depth map at depth/train/scene_001.png.

Dataset YAML format#

Depth estimation datasets are configured with YAML files. The main fields are:

# Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license

# NYU Depth V2 dataset for monocular depth estimation
# Documentation: https://cs.nyu.edu/~fergus/datasets/nyu_depth_v2.html
# 795 train + 654 val (Eigen test split) images, 480x640, indoor scenes, depth in meters
# Example usage: yolo depth train data=nyu-depth.yaml model=yolo26n-depth.pt
# parent
# ├── ultralytics
# └── datasets
#     └── nyu-depth-png  ← downloads here (≈1.5 GB)

# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: nyu-depth-png # dataset root dir (relative to Ultralytics settings 'datasets_dir')
train: images/train # train images (relative to 'path') 795 images
val: images/val # val images (relative to 'path') 654 images

# Depth maps are paired uint16 millimeter PNGs under depth/<split>/, resolved by
# swapping '/images/' -> '/depth/' on each image path.

# Classes
nc: 1
names:
  0: depth

channels: 3
depth_scale: 1000

# Download script/URL (optional)
download: https://github.com/ultralytics/assets/releases/download/v0.0.0/nyu-depth-png.zip

Usage#

Train a YOLO26 depth estimation model with Python or CLI:

from ultralytics import YOLO

# Load a pretrained depth model
model = YOLO("yolo26n-depth.pt")

# Train on the NYU Depth V2 dataset
results = model.train(data="nyu-depth.yaml", epochs=100, imgsz=640)

Supported Datasets#

The YOLO26 depth models are pretrained on a broad multi-dataset mix (~2.19M images) spanning indoor (≤10 m) to outdoor (~80 m) ranges, then evaluated zero-shot across five benchmarks. Each dataset has a dedicated page:

Debugging

• Depth8 — 8 SUN RGB-D images in a 1.3 MB auto-downloading archive, for rapid pipeline testing

Pretraining sources

• ARKitScenes — real indoor, Apple ARKit LiDAR (largest real source)
• SUN RGB-D — real indoor, multi-sensor RGB-D
• DIODE — real indoor + outdoor, dense laser-scanner ground truth
• Hypersim — synthetic photorealistic indoor
• TartanAir — synthetic, diverse environments
• Virtual KITTI 2 — synthetic outdoor driving
• KITTI — real outdoor driving, Velodyne LiDAR (also an evaluation benchmark)
• ImageNet (pseudo-labeled) — pseudo-labeled distillation set, the largest single source

Evaluation benchmarks

• NYU Depth V2 — primary indoor benchmark
• KITTI Eigen — outdoor driving benchmark
• ETH3D — high-precision indoor + outdoor
• Make3D — outdoor, out-of-distribution
• iBims-1 — high-quality indoor (edges and planar surfaces)

Per-model accuracy on these benchmarks and the downloadable pretrained weights are listed on the Depth Estimation task page. A dataset YAML whose train field lists multiple image directories combines these sources for large-scale mixed training.

Adding Your Own Dataset#

1. Save RGB images under split folders such as images/train and images/val.
2. Save one depth PNG or NPY per image under the matching depth/train and depth/val folders using the same file stem as the image. Use save_depth_png() to convert meter arrays into compact millimeter PNGs.
3. Ensure the decoded depth values are in meters and that invalid or missing pixels use 0 or negative values.
4. Create a dataset YAML with path, train, val, nc: 1, and names: {0: depth}.

path: path/to/my-depth-dataset
train: images/train
val: images/val

nc: 1
names:
    0: depth

# Optional: PNG integer units per meter (default 1000)
depth_scale: 1000

FAQ#