Link to this sectionARKitScenes Depth Dataset#

ARKitScenes is a large-scale real-world indoor RGB-D dataset captured with Apple's ARKit on iPad Pro devices equipped with a LiDAR scanner. It is the largest real-world indoor RGB-D dataset of its kind, providing diverse, naturally captured indoor scenes with accurate depth ground truth for monocular depth estimation.

Link to this sectionKey Features#

• Captured with the LiDAR scanner and RGB camera of Apple's ARKit on iPad Pro, producing real (non-synthetic) sensor data.
• Covers diverse indoor scenes for 3D indoor scene understanding.
• Short depth range, approximately 0.5–6 m (median maximum depth around 2.4 m), typical of handheld indoor capture.
• Dense LiDAR-derived depth ground truth aligned to the RGB frames.
• The single largest real-world source in the Ultralytics depth pretraining mix.

Link to this sectionDataset Structure#

The ARKitScenes depth dataset is split into two subsets:

1. Train: 676,080 images with paired depth maps for training.
2. Val: 21,559 images with paired depth maps for validation during model training.

Each sample consists of one RGB image and one paired .npy float32 depth map storing per-pixel distances in meters, following the Ultralytics depth dataset format.

Link to this sectionObtain the Data#

ARKitScenes has no autodownload — the data is distributed by Apple, and downloading requires accepting the license terms in the ARKitScenes repository. Clone the repository and download the depth-upsampling subset, which pairs RGB frames with registered depth:

git clone https://github.com/apple/ARKitScenes && cd ARKitScenes
python3 download_data.py upsampling --split Training --download_dir ./data
python3 download_data.py upsampling --split Validation --download_dir ./data

Depth frames are uint16 PNGs in millimeters (0 = invalid), and RGB/depth filenames are capture timestamps that do not match exactly — pair each depth frame with the nearest-timestamp RGB frame. Reference conversion to the Ultralytics depth dataset format:

from pathlib import Path

import cv2
import numpy as np

src, dst = Path("data/upsampling"), Path("datasets/depth-arkitscenes")
for split, out in (("Training", "train"), ("Validation", "val")):
    (dst / f"images/{out}").mkdir(parents=True, exist_ok=True)
    (dst / f"depth/{out}").mkdir(parents=True, exist_ok=True)
    for video in sorted((src / split).iterdir()):
        rgbs = {float(p.stem.split("_")[-1]): p for p in (video / "lowres_wide").glob("*.png")}
        if not rgbs:
            continue
        for depth_png in sorted((video / "lowres_depth").glob("*.png")):
            t = float(depth_png.stem.split("_")[-1])
            rgb = rgbs[min(rgbs, key=lambda k: abs(k - t))]  # nearest-timestamp RGB frame
            name = f"{video.name}_{depth_png.stem}"
            depth = cv2.imread(str(depth_png), cv2.IMREAD_UNCHANGED).astype(np.float32) / 1000.0  # mm → m
            np.save(dst / f"depth/{out}/{name}.npy", depth)
            cv2.imwrite(str(dst / f"images/{out}/{name}.png"), cv2.imread(str(rgb)))

Link to this sectionRole in YOLO26-Depth#

ARKitScenes is a training source in the Ultralytics YOLO26-Depth multi-dataset pretraining mix of roughly 2.19M image–depth pairs. As the single largest real source in this mix, it supplies abundant real-world indoor LiDAR depth that anchors the model's short-range indoor accuracy. The resulting models are evaluated on the standard NYU, KITTI, Make3D, ETH3D, and iBims-1 benchmarks.

Link to this sectionDataset YAML#

A YAML (Yet Another Markup Language) file is used to define the dataset configuration. It contains information about the dataset's paths, classes, and other relevant information.

# Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license

# ARKitScenes dataset for monocular depth estimation — real indoor RGB-D from Apple ARKit (iPad Pro LiDAR), depth ~0.5-6 m
# Documentation: https://docs.ultralytics.com/datasets/depth/arkitscenes
# Example usage: yolo depth train data=depth-arkitscenes.yaml model=yolo26n-depth.pt
# No autodownload — obtain the source data (see docs) and arrange it as below.
# parent
# ├── ultralytics
# └── datasets
#     └── depth-arkitscenes
#         ├── images/{train,val}  # RGB images
#         └── depth/{train,val}   # paired *.npy, float32 meters (images/ -> depth/)

path: depth-arkitscenes # dataset root dir (relative to Ultralytics settings 'datasets_dir')
train: images/train # train images (relative to 'path') 676080 images
val: images/val # val images (relative to 'path') 21559 images

nc: 1
names:
  0: depth

channels: 3

Link to this sectionUsage#

To train a YOLO26n-depth model on the ARKitScenes dataset with an image size of 640, you can use the following code snippets. For a comprehensive list of available arguments, refer to the model Training page.

from ultralytics import YOLO

# Load a model
model = YOLO("yolo26n-depth.pt")  # load a pretrained depth model (recommended for training)

# Train the model
results = model.train(data="depth-arkitscenes.yaml", epochs=100, imgsz=640)

Link to this sectionPretrained Models#

The YOLO26 depth family is trained on the broad multi-dataset depth pretraining mix that ARKitScenes is part of. These models auto-download from the latest Ultralytics release, for example YOLO26x-depth from v8.4.0, and span a range of sizes for different accuracy and resource requirements.

Link to this sectionCitations and Acknowledgments#

If you use the ARKitScenes dataset in your research or development work, please cite the following paper:

@inproceedings{baruch2021arkitscenes,
      title={ARKitScenes: A Diverse Real-World Dataset for 3D Indoor Scene Understanding Using Mobile RGB-D Data},
      author={Baruch, Gilad and Chen, Zhuoyuan and Dehghan, Afshin and Dimry, Tal and Feigin, Yuri and Fu, Peter and Gebauer, Thomas and Joffe, Brandon and Kurz, Daniel and Schwartz, Arik and Shulman, Elad},
      booktitle={Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
      year={2021}
}

We would like to acknowledge the authors for creating and maintaining this valuable resource for the computer vision community.