Quick Start Guide: NVIDIA DGX Spark with Ultralytics YOLO11

This comprehensive guide provides a detailed walkthrough for deploying Ultralytics YOLO11 on NVIDIA DGX Spark, NVIDIA's compact desktop AI supercomputer. Additionally, it showcases performance benchmarks to demonstrate the capabilities of YOLO11 on this powerful system.

What is NVIDIA DGX Spark?

NVIDIA DGX Spark is a compact desktop AI supercomputer powered by the NVIDIA GB10 Grace Blackwell Superchip. It delivers up to 1 petaFLOP of AI computing performance with FP4 precision, making it ideal for developers, researchers, and data scientists who need powerful AI capabilities in a desktop form factor.

Key Specifications

DGX OS

NVIDIA DGX OS is a customized Linux distribution that provides a stable, tested, and supported operating system foundation for running AI, machine learning, and analytics applications on DGX systems. It includes:

• A robust Linux foundation optimized for AI workloads
• Pre-configured drivers and system settings for NVIDIA hardware
• Security updates and system maintenance capabilities
• Compatibility with the broader NVIDIA software ecosystem

DGX OS follows a regular release schedule with updates typically provided twice per year (around February and August), with additional security patches provided between major releases.

DGX Dashboard

DGX Spark comes with a built-in DGX Dashboard that provides:

• Real-time System Monitoring: Overview of the system's current operational metrics
• System Updates: Ability to apply updates directly from the dashboard
• System Settings: Change device name and other configurations
• Integrated JupyterLab: Access local Jupyter Notebooks for development

Accessing the Dashboard

# Open an SSH tunnel
ssh -L 11000:localhost:11000 <username>@<IP or spark-abcd.local>

# Then open in browser
# http://localhost:11000

Quick Start with Docker

The fastest way to get started with Ultralytics YOLO11 on NVIDIA DGX Spark is to run with pre-built docker images. The same Docker image that supports Jetson AGX Thor (JetPack 7.0) works on DGX Spark with DGX OS.

t=ultralytics/ultralytics:latest-nvidia-arm64
sudo docker pull $t && sudo docker run -it --ipc=host --runtime=nvidia --gpus all $t

After this is done, skip to Use TensorRT on NVIDIA DGX Spark section.

Start with Native Installation

For a native installation without Docker, follow these steps.

Install Ultralytics Package

Here we will install Ultralytics package on DGX Spark with optional dependencies so that we can export the PyTorch models to other different formats. We will mainly focus on NVIDIA TensorRT exports because TensorRT will make sure we can get the maximum performance out of the DGX Spark.

1. Update packages list, install pip and upgrade to latest

   sudo apt update
   sudo apt install python3-pip -y
   pip install -U pip
   

2. Install ultralytics pip package with optional dependencies

   pip install ultralytics[export]
   

3. Reboot the device

   sudo reboot
   

Install PyTorch and Torchvision

The above ultralytics installation will install Torch and Torchvision. However, these packages installed via pip may not be fully optimized for the DGX Spark's ARM64 architecture with CUDA 13. Therefore, we recommend installing the CUDA 13 compatible versions:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu130

UserWarning: Found GPU0 NVIDIA GB10 which is of cuda capability 12.1.
Minimum and Maximum cuda capability supported by this version of PyTorch is (8.0) - (12.0)

Install onnxruntime-gpu

The onnxruntime-gpu package hosted in PyPI does not have aarch64 binaries for ARM64 systems. So we need to manually install this package. This package is needed for some of the exports.

Here we will download and install onnxruntime-gpu 1.24.0 with Python3.12 support.

pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/onnxruntime_gpu-1.24.0-cp312-cp312-linux_aarch64.whl

Use TensorRT on NVIDIA DGX Spark

Among all the model export formats supported by Ultralytics, TensorRT offers the highest inference performance on NVIDIA DGX Spark, making it our top recommendation for deployments. For setup instructions and advanced usage, see our dedicated TensorRT integration guide.

Convert Model to TensorRT and Run Inference

The YOLO11n model in PyTorch format is converted to TensorRT to run inference with the exported model.

from ultralytics import YOLO

# Load a YOLO11n PyTorch model
model = YOLO("yolo11n.pt")

# Export the model to TensorRT
model.export(format="engine")  # creates 'yolo11n.engine'

# Load the exported TensorRT model
trt_model = YOLO("yolo11n.engine")

# Run inference
results = trt_model("https://ultralytics.com/images/bus.jpg")

# Export a YOLO11n PyTorch model to TensorRT format
yolo export model=yolo11n.pt format=engine # creates 'yolo11n.engine'

# Run inference with the exported model
yolo predict model=yolo11n.engine source='https://ultralytics.com/images/bus.jpg'

NVIDIA DGX Spark YOLO11 Benchmarks

YOLO11 benchmarks were run by the Ultralytics team on multiple model formats measuring speed and accuracy: PyTorch, TorchScript, ONNX, OpenVINO, TensorRT, TF SavedModel, TF GraphDef, TF Lite, MNN, NCNN, ExecuTorch. Benchmarks were run on NVIDIA DGX Spark at FP32 precision with default input image size of 640.

Detailed Comparison Table

The below table represents the benchmark results for five different models (YOLO11n, YOLO11s, YOLO11m, YOLO11l, YOLO11x) across multiple formats, giving us the status, size, mAP50-95(B) metric, and inference time for each combination.

Reproduce Our Results

To reproduce the above Ultralytics benchmarks on all export formats run this code:

from ultralytics import YOLO

# Load a YOLO11n PyTorch model
model = YOLO("yolo11n.pt")

# Benchmark YOLO11n speed and accuracy on the COCO128 dataset for all export formats
results = model.benchmark(data="coco128.yaml", imgsz=640)

# Benchmark YOLO11n speed and accuracy on the COCO128 dataset for all export formats
yolo benchmark model=yolo11n.pt data=coco128.yaml imgsz=640

Best Practices for NVIDIA DGX Spark

When using NVIDIA DGX Spark, there are a couple of best practices to follow in order to enable maximum performance running YOLO11.

1. Monitor System Performance

   Use NVIDIA's monitoring tools to track GPU and CPU utilization:

   nvidia-smi
   

2. Optimize Memory Usage

   With 128GB of unified memory, DGX Spark can handle large batch sizes and models. Consider increasing batch size for improved throughput:

   from ultralytics import YOLO
   
   model = YOLO("yolo11n.engine")
   results = model.predict(source="path/to/images", batch=16)
   

3. Use TensorRT with FP16 or INT8

   For best performance, export models with FP16 or INT8 precision:

   yolo export model=yolo11n.pt format=engine half=True # FP16
   yolo export model=yolo11n.pt format=engine int8=True # INT8
   

System Updates (Founders Edition)

Keeping your DGX Spark Founders Edition up to date is crucial for performance and security. NVIDIA provides two primary methods for updating the system OS, drivers, and firmware.

Using DGX Dashboard (Recommended)

The DGX Dashboard is the recommended way to perform system updates ensuring compatibility. It allows you to:

• View available system updates
• Install security patches and system updates
• Manage NVIDIA driver and firmware updates

Manual System Updates

For advanced users, updates can be performed manually via terminal:

sudo apt update
sudo apt dist-upgrade
sudo fwupdmgr refresh
sudo fwupdmgr upgrade
sudo reboot

Next Steps

For further learning and support, see the Ultralytics YOLO11 Docs.

FAQ

How do I deploy Ultralytics YOLO11 on NVIDIA DGX Spark?

Deploying Ultralytics YOLO11 on NVIDIA DGX Spark is straightforward. You can use the pre-built Docker image for quick setup or manually install the required packages. Detailed steps for each approach can be found in sections Quick Start with Docker and Start with Native Installation.

What performance can I expect from YOLO11 on NVIDIA DGX Spark?

YOLO11 models deliver excellent performance on DGX Spark thanks to the GB10 Grace Blackwell Superchip. The TensorRT format provides the best inference performance. Check the Detailed Comparison Table section for specific benchmark results across different model sizes and formats.

Why should I use TensorRT for YOLO11 on DGX Spark?

TensorRT is highly recommended for deploying YOLO11 models on DGX Spark due to its optimal performance. It accelerates inference by leveraging the Blackwell GPU capabilities, ensuring maximum efficiency and speed. Learn more in the Use TensorRT on NVIDIA DGX Spark section.

How does DGX Spark compare to Jetson devices for YOLO11?

DGX Spark offers significantly more compute power than Jetson devices with up to 1 PFLOP of AI performance and 128GB unified memory, compared to Jetson AGX Thor's 2070 TFLOPS and 128GB memory. DGX Spark is designed as a desktop AI supercomputer, while Jetson devices are embedded systems optimized for edge deployment.

Can I use the same Docker image for DGX Spark and Jetson AGX Thor?

Yes! The ultralytics/ultralytics:latest-nvidia-arm64 Docker image supports both NVIDIA DGX Spark (with DGX OS) and Jetson AGX Thor (with JetPack 7.0), as both use ARM64 architecture with CUDA 13 and similar software stacks.

📅 Created 0 days ago ✏️ Updated 0 days ago

Comments