Ultralytics YOLO27: Join the waitlist.
Get Started

Quick Start Guide: NVIDIA Jetson with Ultralytics YOLO26#

This guide walks through deploying Ultralytics YOLO26 on NVIDIA Jetson devices and includes performance benchmarks that show what YOLO26 can do on these compact, power-efficient devices.

New product support

We have updated this guide with the latest NVIDIA Jetson AGX Thor Developer Kit which delivers up to 2070 FP4 TFLOPS of AI compute and 128 GB of memory with power configurable between 40 W and 130 W. It delivers over 7.5x higher AI compute than NVIDIA Jetson AGX Orin, with 3.5x better energy efficiency for running the most popular AI models.



Watch: Unlock Faster Inference with Ultralytics YOLO26 on NVIDIA Jetson Devices 🚀
NVIDIA Jetson Ecosystem
Note

This guide has been tested with NVIDIA Jetson AGX Thor Developer Kit (Jetson T5000) and NVIDIA Jetson AGX Orin Developer Kit (64GB) running the latest stable JetPack 7.2, NVIDIA Jetson Orin Nano Super Developer Kit running JetPack release of JP6.1, Seeed Studio reComputer J4012 which is based on NVIDIA Jetson Orin NX 16GB running JetPack release of JP6.0/ JetPack release of JP5.1.3 and Seeed Studio reComputer J1020 v2 which is based on NVIDIA Jetson Nano 4GB running JetPack release of JP4.6.1. It is expected to work across all the NVIDIA Jetson hardware lineup, including the latest and legacy devices.

What is NVIDIA Jetson?#

NVIDIA Jetson is a series of embedded computing boards designed to bring accelerated AI (artificial intelligence) computing to edge devices. These compact and powerful devices are built around NVIDIA's GPU architecture and can run complex AI algorithms and deep learning models directly on the device, without relying on cloud computing resources. Jetson boards are often used in robotics, autonomous vehicles, industrial automation, and other applications where AI inference needs to be performed locally with low latency and high efficiency. Additionally, these boards are based on the ARM64 architecture and run at lower power compared to traditional GPU computing devices.

NVIDIA Jetson Series Comparison#

NVIDIA Jetson AGX Thor is the latest iteration of the NVIDIA Jetson family based on the NVIDIA Blackwell architecture, which delivers significantly higher AI performance than previous generations. The table below compares a few of the Jetson devices in the ecosystem.

Jetson AGX Thor (T5000)Jetson AGX Orin 64GBJetson Orin NX 16GBJetson Orin Nano SuperJetson AGX XavierJetson Xavier NXJetson Nano
AI Performance2070 TFLOPS275 TOPS100 TOPS67 TOPS32 TOPS21 TOPS472 GFLOPS
GPU2560-core NVIDIA Blackwell architecture GPU with 96 Tensor Cores2048-core NVIDIA Ampere architecture GPU with 64 Tensor Cores1024-core NVIDIA Ampere architecture GPU with 32 Tensor Cores1024-core NVIDIA Ampere architecture GPU with 32 Tensor Cores512-core NVIDIA Volta architecture GPU with 64 Tensor Cores384-core NVIDIA Volta™ architecture GPU with 48 Tensor Cores128-core NVIDIA Maxwell™ architecture GPU
GPU Max Frequency1.57 GHz1.3 GHz918 MHz1020 MHz1377 MHz1100 MHz921MHz
CPU14-core Arm® Neoverse®-V3AE 64-bit CPU 1MB L2 + 16MB L312-core NVIDIA Arm® Cortex A78AE v8.2 64-bit CPU 3MB L2 + 6MB L38-core NVIDIA Arm® Cortex A78AE v8.2 64-bit CPU 2MB L2 + 4MB L36-core Arm® Cortex®-A78AE v8.2 64-bit CPU 1.5MB L2 + 4MB L38-core NVIDIA Carmel Arm®v8.2 64-bit CPU 8MB L2 + 4MB L36-core NVIDIA Carmel Arm®v8.2 64-bit CPU 6MB L2 + 4MB L3Quad-Core Arm® Cortex®-A57 MPCore processor
CPU Max Frequency2.6 GHz2.2 GHz2.0 GHz1.7 GHz2.2 GHz1.9 GHz1.43GHz
Memory128GB 256-bit LPDDR5X 273GB/s64GB 256-bit LPDDR5 204.8GB/s16GB 128-bit LPDDR5 102.4GB/s8GB 128-bit LPDDR5 102 GB/s32GB 256-bit LPDDR4x 136.5GB/s8GB 128-bit LPDDR4x 59.7GB/s4GB 64-bit LPDDR4 25.6GB/s

What is NVIDIA JetPack?#

NVIDIA JetPack SDK powers the Jetson modules and provides a full development environment for building end-to-end accelerated AI applications. JetPack includes Jetson Linux with bootloader, Linux kernel, Ubuntu desktop environment, and a complete set of libraries for acceleration of GPU computing, multimedia, graphics, and computer vision. It also includes samples, documentation, and developer tools for both host computer and developer kit, and supports higher level SDKs such as DeepStream for streaming video analytics, Isaac for robotics, and Riva for conversational AI.

Flash JetPack to NVIDIA Jetson#

The first step after getting your hands on an NVIDIA Jetson device is to flash NVIDIA JetPack to the device. There are several different ways of flashing NVIDIA Jetson devices.

  1. For JetPack 7.2 on an official Jetson AGX Thor, AGX Orin, or Orin Nano Developer Kit, download the unified Jetson ISO, write it to a USB flash drive, and follow the device-specific quick start for AGX Thor, AGX Orin, or Orin Nano. Starting with JetPack 7.2, Orin Nano no longer uses a downloadable SD-card image; the ISO USB installs Jetson Linux onto the device's microSD card or NVMe SSD.
  2. If you intentionally use JetPack 6 on a Jetson Orin Nano Developer Kit, follow NVIDIA's JetPack 6.x update and SD-card instructions.
  3. If you own any other NVIDIA Development Kit, you can flash JetPack to the device using SDK Manager.
  4. If you own a Seeed Studio reComputer J4012 device, you can flash JetPack to the included SSD and if you own a Seeed Studio reComputer J1020 v2 device, you can flash JetPack to the eMMC/ SSD.
  5. If you own any other third-party device powered by the NVIDIA Jetson module, it is recommended to follow command-line flashing.
Note

For methods 1, 4 and 5 above, after flashing the system and booting the device, run sudo apt update && sudo apt install nvidia-jetpack -y in the device terminal to install all the remaining JetPack components needed.

JetPack Support Based on Jetson Device#

The below table highlights NVIDIA JetPack versions supported by different NVIDIA Jetson devices.

JetPack 4JetPack 5JetPack 6JetPack 7
Jetson Nano✅❌❌❌
Jetson TX2✅❌❌❌
Jetson Xavier NX✅✅❌❌
Jetson AGX Xavier✅✅❌❌
Jetson AGX Orin❌✅✅✅
Jetson Orin NX❌✅✅✅
Jetson Orin Nano❌✅✅✅
Jetson AGX Thor❌❌❌✅

Quick Start with Docker#

The fastest way to get started with Ultralytics YOLO26 on NVIDIA Jetson is to run the pre-built Docker images for Jetson. Refer to the table above and choose the JetPack version according to the Jetson device you own.

t=ultralytics/ultralytics:latest-jetson-jetpack4
sudo docker pull $t && sudo docker run -it --ipc=host --runtime=nvidia $t
JetPack 7 Docker and TensorRT support

The latest-nvidia-arm64 image uses NVIDIA PyTorch 26.08, including CUDA 13.4 and TensorRT 11.2. NVIDIA lists JetPack 7.1 for PyTorch in its compatibility table, but TensorRT 11.2.1 does not support JetPack, including Thor. Use this image only for PyTorch workloads on a supported host. For Jetson TensorRT exports, use the native installation below with the TensorRT 10.x runtime supported by your JetPack release. JetPack 7.2 on Thor or Orin requires separate container validation. Rebuild engines for the target GPU and TensorRT version.

For JetPack 4, 5, and 6 images, continue to Use TensorRT on NVIDIA Jetson. The JetPack 7.1 image above is for PyTorch workloads only.

Start with Native Installation#

For a native installation without Docker, please refer to the steps below.

Run on JetPack 7.2#

Install Ultralytics Package#

Install the Ultralytics package on the Jetson. Export dependencies are installed automatically the first time you export a model, so only the base package is needed here. This guide focuses on NVIDIA TensorRT exports because TensorRT delivers the best performance on Jetson devices.

  1. Update packages list, install pip and upgrade to latest

    sudo apt update
sudo apt install python3-pip -y
pip install -U pip

  2. Install ultralytics pip package

    pip install ultralytics

  3. Reboot the device

    sudo reboot

Install PyTorch and Torchvision#

The above ultralytics installation will install Torch and Torchvision. However, these two packages installed via pip are not compatible with JetPack 7.2 devices running CUDA 13, so they need to be installed manually.

Install torch and torchvision according to JP7.2

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu130

Install onnxruntime-gpu#

Use an ONNX Runtime GPU wheel matched to your JetPack, Python, and CUDA versions. Current PyPI releases include ARM64 wheels, but they do not replace the device-specific packages for every JetPack release.

Download and install onnxruntime-gpu 1.24.0 with Python3.12 support.

pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/onnxruntime_gpu-1.24.0-cp312-cp312-linux_aarch64.whl

Run on JetPack 6.1#

Install Ultralytics Package#

Install the Ultralytics package on the Jetson. Export dependencies are installed automatically the first time you export a model, so only the base package is needed here. This guide focuses on NVIDIA TensorRT exports because TensorRT delivers the best performance on Jetson devices.

  1. Update packages list, install pip and upgrade to latest

    sudo apt update
sudo apt install python3-pip -y
pip install -U pip

  2. Install ultralytics pip package

    pip install ultralytics

  3. Reboot the device

    sudo reboot

Install PyTorch and Torchvision#

The above ultralytics installation will install Torch and Torchvision. However, these two packages installed via pip are not compatible with the Jetson platform, which is based on ARM64 architecture, so you need to manually install a pre-built PyTorch pip wheel and compile or install Torchvision from source.

Install torch 2.10.0 and torchvision 0.25.0 according to JP6.1

pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/torch-2.10.0-cp310-cp310-linux_aarch64.whl
pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/torchvision-0.25.0-cp310-cp310-linux_aarch64.whl
Note

Visit the PyTorch for Jetson page to access all different versions of PyTorch for different JetPack versions. For a more detailed list on the PyTorch, Torchvision compatibility, visit the PyTorch and Torchvision compatibility page.

Install cuDSS to fix a dependency issue with torch 2.10.0

wget https://developer.download.nvidia.com/compute/cudss/0.7.1/local_installers/cudss-local-tegra-repo-ubuntu2204-0.7.1_0.7.1-1_arm64.deb
sudo dpkg -i cudss-local-tegra-repo-ubuntu2204-0.7.1_0.7.1-1_arm64.deb
sudo cp /var/cudss-local-tegra-repo-ubuntu2204-0.7.1/cudss-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cudss

Install onnxruntime-gpu#

Use an ONNX Runtime GPU wheel matched to your JetPack, Python, and CUDA versions. Current PyPI releases include ARM64 wheels, but they do not replace the device-specific packages for every JetPack release.

You can find all available onnxruntime-gpu packages—organized by JetPack version, Python version, and other compatibility details—in the Jetson Zoo ONNX Runtime compatibility matrix.

For JetPack 6 with Python 3.10 support, you can install onnxruntime-gpu 1.23.0:

pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/onnxruntime_gpu-1.23.0-cp310-cp310-linux_aarch64.whl

Alternatively, for onnxruntime-gpu 1.20.0:

pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/onnxruntime_gpu-1.20.0-cp310-cp310-linux_aarch64.whl

Run on JetPack 5.1.2#

Install Ultralytics Package#

Install the Ultralytics package on the Jetson. Export dependencies are installed automatically the first time you export a model, so only the base package is needed here. This guide focuses on NVIDIA TensorRT exports because TensorRT delivers the best performance on Jetson devices.

  1. Update packages list, install pip and upgrade to latest

    sudo apt update
sudo apt install python3-pip -y
pip install -U pip

  2. Install ultralytics pip package

    pip install ultralytics

  3. Reboot the device

    sudo reboot

Install PyTorch and Torchvision#

The above ultralytics installation will install Torch and Torchvision. However, these two packages installed via pip are not compatible with the Jetson platform, which is based on ARM64 architecture, so you need to manually install a pre-built PyTorch pip wheel and compile or install Torchvision from source.

  1. Uninstall currently installed PyTorch and Torchvision

    pip uninstall torch torchvision

  2. Install torch 2.1.0 and torchvision 0.16.2 according to JP5.1.2

    pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl
pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/torchvision-0.16.2+c6f3977-cp38-cp38-linux_aarch64.whl
Note

Visit the PyTorch for Jetson page to access all different versions of PyTorch for different JetPack versions. For a more detailed list on the PyTorch, Torchvision compatibility, visit the PyTorch and Torchvision compatibility page.

Install onnxruntime-gpu#

Use an ONNX Runtime GPU wheel matched to your JetPack, Python, and CUDA versions. Current PyPI releases include ARM64 wheels, but they do not replace the device-specific packages for every JetPack release.

You can find all available onnxruntime-gpu packages—organized by JetPack version, Python version, and other compatibility details—in the Jetson Zoo ONNX Runtime compatibility matrix. Download and install onnxruntime-gpu 1.17.0 with Python3.8 support.

wget https://nvidia.box.com/shared/static/zostg6agm00fb6t5uisw51qi6kpcuwzd.whl -O onnxruntime_gpu-1.17.0-cp38-cp38-linux_aarch64.whl
pip install onnxruntime_gpu-1.17.0-cp38-cp38-linux_aarch64.whl
Note

onnxruntime-gpu automatically upgrades NumPy to the latest version, so reinstall NumPy 1.23.5 to fix the resulting issue:

pip install numpy==1.23.5

Use TensorRT on NVIDIA Jetson#

Among all the model export formats supported by Ultralytics, TensorRT offers the highest inference performance on NVIDIA Jetson devices, making it our top recommendation for Jetson deployments. Before exporting, install the TensorRT Python bindings supplied for your JetPack release and verify them with python3 -c "import tensorrt; print(tensorrt.__version__)". Keep the supported TensorRT 10.x runtime on JetPack 6/7; do not replace it with TensorRT 11.2.1. For setup instructions and advanced usage, see our dedicated TensorRT integration guide.

You can also export from the browser without configuring the build environment locally. In the Ultralytics Platform model Export tab, select TensorRT and the intended Jetson target. Thor selections are validated on physical Thor hardware. The six Orin selections currently produce AGX-Orin-built candidate engines; validate them on the intended Orin SKU before deployment.

TensorRT engines are device-specific

TensorRT profiles and tunes an engine on its build GPU. Match the target's GPU architecture and TensorRT/CUDA runtime, and validate every downloaded engine on the deployment device. Same-architecture Orin SKUs are not an automatic portability guarantee, and INT8 calibration should use the target device for best results.

Convert Model to TensorRT and Run Inference#

The YOLO26n model in PyTorch format is converted to TensorRT to run inference with the exported model.

Example
from ultralytics import YOLO

# Load a YOLO26n PyTorch model
model = YOLO("yolo26n.pt")

# Export the model to TensorRT
model.export(format="engine")  # creates 'yolo26n.engine'

# Load the exported TensorRT model
trt_model = YOLO("yolo26n.engine")

# Run inference
results = trt_model("https://ultralytics.com/images/bus.jpg")

Use NVIDIA Deep Learning Accelerator (DLA)#

NVIDIA Deep Learning Accelerator (DLA) is a specialized hardware component built into NVIDIA Jetson devices that optimizes deep learning inference for energy efficiency and performance. By offloading tasks from the GPU (freeing it up for more intensive processes), DLA enables models to run with lower power consumption while maintaining high throughput, ideal for embedded systems and real-time AI applications.

TensorRT and DLA compatibility

NVIDIA lists TensorRT 10.7 as the last release with DLA support; TensorRT 11.x does not support DLA. Use a DLA-capable TensorRT release supported by your JetPack version. TensorRT 11.2.1 does not support JetPack, including GPU-only exports.

The following Jetson devices are equipped with DLA hardware:

Jetson DeviceDLA CoresDLA Max Frequency
Jetson AGX Orin Series21.6 GHz
Jetson Orin NX 16GB2614 MHz
Jetson Orin NX 8GB1614 MHz
Jetson AGX Xavier Series21.4 GHz
Jetson Xavier NX Series21.1 GHz
Example
from ultralytics import YOLO

# Load a YOLO26n PyTorch model
model = YOLO("yolo26n.pt")

# Export the model to TensorRT with DLA enabled (only works with FP16 or INT8)
model.export(format="engine", device="dla:0", quantize=16)  # dla:0 or dla:1 corresponds to the DLA cores

# Load the exported TensorRT model
trt_model = YOLO("yolo26n.engine")

# Run inference
results = trt_model("https://ultralytics.com/images/bus.jpg")
Note

When using DLA exports, some layers may not be supported to run on DLA and will fall back to the GPU for execution. This fallback can introduce additional latency and impact the overall inference performance. Therefore, DLA is not primarily designed to reduce inference latency compared to TensorRT running entirely on the GPU. Instead, its primary purpose is to increase throughput and improve energy efficiency.

NVIDIA Jetson YOLO26 Benchmarks#

YOLO26 benchmarks were run by the Ultralytics team on 11 different model formats measuring speed and accuracy: PyTorch, TorchScript, ONNX, OpenVINO, TensorRT, TF SavedModel, TF GraphDef, TF Lite, MNN, NCNN, ExecuTorch. Benchmarks were run on NVIDIA Jetson AGX Thor Developer Kit, NVIDIA Jetson AGX Orin Developer Kit (64GB), NVIDIA Jetson Orin Nano Super Developer Kit and Seeed Studio reComputer J4012 powered by Jetson Orin NX 16GB device at FP32 precision with the default input image size of 640.

Comparison Charts#

Even though all model exports work on NVIDIA Jetson, the comparison chart below includes only PyTorch, TorchScript, and TensorRT. You can find benchmarks for all exports in the section after this chart.

NVIDIA Jetson AGX Thor Developer Kit#

Jetson AGX Thor Benchmarks
Benchmarked with Ultralytics 8.3.226

NVIDIA Jetson AGX Orin Developer Kit (64GB)#

Jetson AGX Orin Benchmarks
Benchmarked with Ultralytics 8.4.32

NVIDIA Jetson Orin Nano Super Developer Kit#

Jetson Orin Nano Super Benchmarks
Benchmarked with Ultralytics 8.4.33

NVIDIA Jetson Orin NX 16GB#

Jetson Orin NX 16GB Benchmarks
Benchmarked with Ultralytics 8.4.33

Detailed Comparison Tables#

The table below shows the benchmark results for five different YOLO26 models (YOLO26n, YOLO26s, YOLO26m, YOLO26l, YOLO26x) across 11 different formats (PyTorch, TorchScript, ONNX, OpenVINO, TensorRT, TF SavedModel, TF GraphDef, TF Lite, MNN, NCNN, ExecuTorch), giving the status, size, mAP50-95(B) metric, and inference time for each combination.

NVIDIA Jetson AGX Thor Developer Kit#

Performance
FormatStatusSize on disk (MB)mAP50-95(B)Inference time (ms/im)
PyTorch✅5.30.47987.39
TorchScript✅9.80.47894.21
ONNX✅9.50.47676.58
OpenVINO✅10.10.479417.50
TensorRT (FP32)✅13.90.47911.90
TensorRT (FP16)✅7.60.47971.39
TensorRT (INT8)✅6.50.42731.52
TF SavedModel✅25.70.476447.24
TF GraphDef✅9.50.476445.98
TF Lite✅9.90.4764182.04
MNN✅9.40.478421.83

Benchmarked with Ultralytics 8.4.7

Note

Inference time does not include pre/post-processing.

NVIDIA Jetson AGX Orin Developer Kit (64GB)#

Performance
FormatStatusSize on disk (MB)mAP50-95(B)Inference time (ms/im)
PyTorch✅5.30.479011.58
TorchScript✅9.80.47704.60
ONNX✅9.50.47709.87
OpenVINO✅9.60.482028.80
TensorRT (FP32)✅11.50.47704.18
TensorRT (FP16)✅7.90.47892.62
TensorRT (INT8)✅5.40.46402.30
TF SavedModel✅24.60.476071.10
TF GraphDef✅9.50.476070.02
TF Lite✅9.90.4760227.94
MNN✅9.40.476032.46
NCNN✅9.30.481029.93

Benchmarked with Ultralytics 8.4.32

Note

Inference time does not include pre/post-processing.

NVIDIA Jetson Orin Nano Super Developer Kit#

Performance
FormatStatusSize on disk (MB)mAP50-95(B)Inference time (ms/im)
PyTorch✅5.30.479015.60
TorchScript✅9.80.477012.60
ONNX✅9.50.476015.76
OpenVINO✅9.60.482056.23
TensorRT (FP32)✅11.30.47707.53
TensorRT (FP16)✅8.10.48004.57
TensorRT (INT8)✅5.30.44903.80
TF SavedModel✅24.60.4760118.33
TF GraphDef✅9.50.4760116.30
TF Lite✅9.90.4760286.00
MNN✅9.40.476068.77
NCNN✅9.30.481047.50

Benchmarked with Ultralytics 8.4.33

Note

Inference time does not include pre/post-processing.

NVIDIA Jetson Orin NX 16GB#

Performance
FormatStatusSize on disk (MB)mAP50-95(B)Inference time (ms/im)
PyTorch✅5.30.479913.90
TorchScript✅9.80.478711.60
ONNX✅9.50.476314.18
OpenVINO✅9.60.481940.19
TensorRT (FP32)✅11.40.47707.01
TensorRT (FP16)✅8.00.47894.13
TensorRT (INT8)✅5.50.44893.49
TF SavedModel✅24.60.476492.34
TF GraphDef✅9.50.476492.06
TF Lite✅9.90.4764254.43
MNN✅9.40.476048.55
NCNN✅9.30.480534.31

Benchmarked with Ultralytics 8.4.33

Note

Inference time does not include pre/post-processing.

Explore more benchmarking efforts by Seeed Studio running on different versions of NVIDIA Jetson hardware.

Reproduce Our Results#

To reproduce the above Ultralytics benchmarks on all export formats run this code:

Example
from ultralytics import YOLO

# Load a YOLO26n PyTorch model
model = YOLO("yolo26n.pt")

# Benchmark YOLO26n speed and accuracy on the COCO128 dataset for all export formats
results = model.benchmark(data="coco128.yaml", imgsz=640)

Note that benchmarking results might vary based on the exact hardware and software configuration of a system, as well as the current workload of the system at the time the benchmarks are run. For the most reliable results, use a dataset with a large number of images, e.g., data='coco.yaml' (5000 val images).

Best Practices when using NVIDIA Jetson#

Follow these best practices to get maximum performance from YOLO26 on NVIDIA Jetson.

  1. Enable MAX Power Mode

    Enabling MAX Power Mode on the Jetson makes sure all CPU and GPU cores are turned on.

    sudo nvpmodel -m 0

  2. Enable Jetson Clocks

    Enabling Jetson Clocks makes sure all CPU and GPU cores are clocked at their maximum frequency.

    sudo jetson_clocks

  3. Install Jetson Stats Application

    Use the jetson-stats application to monitor the temperatures of system components, view CPU, GPU, and RAM utilization, change power modes, set max clocks, and check JetPack information.

    sudo apt update
sudo pip install jetson-stats
sudo reboot
jtop
jtop dashboard from jetson-stats showing Jetson CPU, GPU, and memory utilization

Memory Optimization Tips for NVIDIA Jetson#

Available memory is often the limiting factor on Jetson devices, particularly on lower-memory variants such as the Jetson Orin Nano (8 GB) or Orin NX 8 GB. The tips below are practical, low-risk changes that can collectively free several hundred megabytes and let you run larger YOLO models or support additional parallel workloads. For a comprehensive treatment see the NVIDIA blog on maximizing memory efficiency on Jetson.

1. Switch to Headless (No-GUI) Boot#

If your Jetson is connected over SSH or running as a production appliance without a display attached, eliminating the desktop environment and display server can recover up to 865 MB of RAM:

sudo systemctl set-default multi-user.target
sudo reboot

To restore the desktop later:

sudo systemctl set-default graphical.target
sudo reboot

2. Disable Unused System Services#

Non-essential background services (Bluetooth, connectivity managers, unused hardware daemons) consume around 32 MB combined. List active services and disable anything your deployment doesn't require:

# List running services
systemctl list-units --type=service --state=running

# Disable a service
sudo systemctl disable SERVICE_NAME

3. Profile Memory Usage#

Before optimizing, identify which processes are actually consuming RAM. procrank sorts processes by PSS (Proportional Set Size), which reflects the true per-process memory footprint more accurately than RSS (Resident Set Size, the total physical RAM pages mapped by a process, including pages shared with other processes):

git clone https://github.com/csimmonds/procrank_linux.git
cd procrank_linux && make
sudo ./procrank

To see per-process GPU and NvMap (CUDA/video pipeline) allocations:

sudo cat /sys/kernel/debug/nvmap/iovmm/clients

4. Run Inference Without a Display in Production#

For inference pipelines that have no live-preview requirement, disabling display-related components (Tiler, OSD, DisplaySink) can save 200+ MB from the pipeline alone. With Ultralytics YOLO, suppress the viewer and write results to disk instead:

Example
from ultralytics import YOLO

model = YOLO("yolo26n.engine")

# show=False prevents any display window; save=True writes annotated output to disk
results = model.predict(source="video.mp4", show=False, save=True)

Cumulative Impact#

OptimizationApprox. Memory Saved
Disable desktop GUI~865 MB
Disable unused OS services~32 MB
Headless inference pipeline (no display)~200+ MB
Total (easy wins)~1 GB+

Combining these changes is especially valuable when targeting TensorRT INT8 models on memory-constrained devices — it can be the difference between fitting a larger model variant in memory or not.

Next Steps#

For further learning and support, see the Ultralytics YOLO26 Docs.

FAQ#

  • To deploy Ultralytics YOLO26 on NVIDIA Jetson devices, first flash your Jetson device with the NVIDIA JetPack SDK. Then, either use a pre-built Docker image for quick setup or manually install the required packages.

  • YOLO26 models have been benchmarked on NVIDIA Jetson AGX Thor, AGX Orin, Orin Nano Super, and Orin NX devices, and the TensorRT format delivers the best inference performance. The Detailed Comparison Tables section lists mAP50-95 and inference time for each model and export format.

  • TensorRT is recommended for deploying YOLO26 models on NVIDIA Jetson because it delivers the fastest inference of all export formats by optimizing the model for the Jetson's GPU.

  • To install PyTorch and Torchvision on NVIDIA Jetson, first uninstall any existing versions that may have been installed via pip. Then, manually install the compatible PyTorch and Torchvision versions for the Jetson's ARM64 architecture. Detailed instructions for this process are provided in the Install PyTorch and Torchvision section.

  • To maximize performance on NVIDIA Jetson with YOLO26, follow these best practices:

    1. Enable MAX Power Mode to utilize all CPU and GPU cores.
    2. Enable Jetson Clocks to run all cores at their maximum frequency.
    3. Install the Jetson Stats application for monitoring system metrics.

    For commands and additional details, refer to the Best Practices when using NVIDIA Jetson section.

  • Available RAM is often the bottleneck on lower-memory Jetson devices. Three easy wins that together can recover over 1 GB:

    1. Switch to headless boot (sudo systemctl set-default multi-user.target) to eliminate the desktop GUI (~865 MB saved).
    2. Disable unused services such as Bluetooth or connectivity managers (~32 MB saved).
    3. Run inference without a display by setting show=False in your YOLO predict call, which avoids allocating display pipeline memory (~200+ MB saved).

    Use procrank to profile per-process RAM usage and sudo cat /sys/kernel/debug/nvmap/iovmm/clients to inspect GPU allocations. See the Memory Optimization Tips section for full details.

  • TensorRT 10.3.0 shipped with JetPack 6 has a known issue that prevents building INT8 engines with NMS-free (nms=False) or embedded NMS (nms=True) outputs. When Ultralytics detects this combination, it selects the one-to-many head for NMS-free exports and stops nms=True exports with an error.

    To restore these INT8 exports, upgrade TensorRT to a newer version (e.g., 10.7.0+):

    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/arm64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get install -y tensorrt

    After upgrading, re-run your export. For more details, see GitHub issue #23841 and issue #26655.

Contributors
GLglenn-jocher32LAlakshanthad28RIRizwanMunawar5ONonuralpszr4LALaughing-q2Y-Y-T-G2RAraimbekovm2BUBurhan-Q2JOJohnny Wilson Dougherty1PDpderrenger1PIpicsalex1JUjustincdavis1MAMatthewNoyce1AHAhelsamahy1

Comments194

Ask a question, share your results, or help someone else.

  2. pknowlerpknowlerGitHub

    Struggling with this on a new Jetson Orin Nano JP6.1. All good following the steps and installs to give torch.cuda.is_available as TRUE. The TensorRT export fails though with a number of dependency errors and I can’t re-install it with anything. Any pointers please?

    1
    1. Glenn JocherGlenn JocherUltralytics

      @pknowler it seems like the dependency errors during TensorRT export might stem from compatibility issues. Please ensure you’ve installed the required JetPack-compatible versions of PyTorch, Torchvision, and TensorRT as outlined in the Jetson Quick Start Guide. Additionally, verify that onnxruntime-gpu and cuSPARSELt are correctly installed per the guide. If the issue persists, you can try reinstalling TensorRT using the NVIDIA SDK Manager for your JetPack version. Let me know if you need further clarification!

    2. pknowlerpknowlerGitHub

      Thank you. I will try again working through all of the steps and let you know how it goes!

    3. pknowlerpknowlerGitHub

      I’ve tried again but this as far as I get… I have followed everything carefully.

      WARNING ⚠️ TensorRT requires GPU export, automatically assigning device=0 Ultralytics 8.3.57 🚀 Python-3.10.16 torch-2.5.0a0+872d972e41.nv24.08 CUDA:0 (Orin, 7620MiB) YOLO11n summary (fused): 238 layers, 2,616,248 parameters, 0 gradients, 6.5 GFLOPs

      PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB)

      ONNX: starting export with onnx 1.17.0 opset 19... ONNX: slimming with onnxslim 0.1.46... ONNX: export success ✅ 3.1s, saved as 'yolo11n.onnx' (10.2 MB) requirements: Ultralytics requirement ['tensorrt>7.0.0,!=10.1.0'] not found, attempting AutoUpdate... error: subprocess-exited-with-error

      × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [6 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/tmp/pip-install-d0ljzl60/tensorrt-cu12_db0552ac431148489703f6f75df6764f/setup.py", line 71, in raise RuntimeError("TensorRT does not currently build wheels for Tegra systems") RuntimeError: TensorRT does not currently build wheels for Tegra systems [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

      × Encountered error while generating package metadata. ╰─> See above for output.

      note: This is an issue with the package mentioned above, not pip. hint: See above for details. Retry 1/2 failed: Command 'pip install --no-cache-dir "tensorrt>7.0.0,!=10.1.0" ' returned non-zero exit status 1. error: subprocess-exited-with-error

      × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [6 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/tmp/pip-install-782zjqf9/tensorrt-cu12_44d261aea7c5489db3e9ce15cec8ee64/setup.py", line 71, in raise RuntimeError("TensorRT does not currently build wheels for Tegra systems") RuntimeError: TensorRT does not currently build wheels for Tegra systems [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

      × Encountered error while generating package metadata. ╰─> See above for output.

      note: This is an issue with the package mentioned above, not pip. hint: See above for details.

    5. Glenn JocherGlenn JocherUltralytics

      @pknowler it seems your TensorRT version is not compatible with the Jetson's ARM64 architecture. TensorRT does not provide pre-built Python wheels for Tegra systems. Please ensure you have TensorRT installed via the NVIDIA SDK Manager as part of the JetPack installation, and confirm compatibility with your JetPack version. The Jetson Quick Start Guide might also help clarify setup specifics. Let me know if further assistance is needed!

    6. pknowlerpknowlerGitHub

      @glenn-jocher thank you. I have found the issue now. I am using miniconda and for some reason TensorRT Python bindings are not available in conda envs. So I can import Torch with no problem but the import TensorRT fails. I have re-installed without conda and all works as expected! 👍🏻

    1. Paula DerrengerPaula DerrengerUltralytics(edited)

      In terms of reducing inference time on your NVIDIA Jetson device, using TensorRT is generally recommended. TensorRT is specifically designed to provide high performance for NVIDIA hardware by optimizing deep learning models for inference. It accelerates models by fusing layers, pruning unused layers, and quantizing variables, among other optimizations. 🚀

      DeepStream, on the other hand, is more focused on streaming analytics and can utilize TensorRT-optimized models. For most cases, directly using TensorRT for model optimization is your go-to for minimizing inference times.

      Here's a quick example of exporting a YOLO model to TensorRT and running inference, tailored for your convenience:

      from ultralytics import YOLO

# Load your YOLO model
model = YOLO('yolo26n.pt')

# Export to TensorRT format
model.export(format='engine')

# Load the TensorRT model
trt_model = YOLO('yolo26n.engine')

# Run inference using the TensorRT model
results = trt_model('your_image_or_video.jpg')

      Feel free to explore more about deploying on NVIDIA Jetson with our Quick Start Guide for NVIDIA Jetson. Happy inferencing! 🌟

      2
    3. Paula DerrengerPaula DerrengerUltralytics(edited)

      Hey there! 😊 It looks like you're interested in speeding up your inference times. Between TensorRT and DeepStream on NVIDIA Jetson devices, TensorRT typically offers more direct optimization for reducing inference time for individual models by providing lower-level optimizations and quantization options.

      DeepStream is a comprehensive framework for building end-to-end video analytics applications, which utilizes TensorRT under the hood for model inference. DeepStream excels in scenarios involving video streams, multi-stream input, and complex pipelines involving multiple AI models and analytics. If your primary goal is solely to minimize inference time for a specific model, direct optimization via TensorRT can provide more focused improvements.

      A quick example of exporting a YOLO model to TensorRT:

      from ultralytics import YOLO

# Load your YOLO model
model = YOLO('yolo26n.pt')

# Export to TensorRT format
model.export(format='engine')  # This creates 'yolo26n.engine'

# Done! Now you can use 'yolo26n.engine' for faster inference.

      However, if your application benefits from a broader video analytics pipeline, incorporating DeepStream into your workflow can optimize and streamline these processes on top of efficiently handling inference with TensorRT optimized models. 🚀

      Hope this gives you a clearer picture! If you have more questions, feel free to ask.

      1
  4. Yufeng WuYufeng Wu(edited)

    Hi, ultralytics team. I'm trying yolo26n-pose on Jetson orin agx 64gb and have this problem: I tried building engine with dynamic=true but failed, with the log saying ForeignNode[/model.23/Sigmoid.../model.23/Flatten] (Myelin) Skipping tactic ... Incompatible effective shapes in operation /model_23/Sub between: Unsqueeze_18_output_0 f32[1,2,33600] and Slice_output_0 f32[1,2,21504] Could not find any implementation for node {ForeignNode[/model.23/Sigmoid.../model.23/Flatten]} Is there a way to fix it?

    1. Glenn JocherGlenn JocherUltralytics

      This indicates a TensorRT dynamic-shape profile failure in the pose head. As a workaround, export a fixed-resolution engine, e.g. yolo export model=yolo26n-pose.pt format=engine imgsz=640 batch=1 dynamic=False; build separate engines if multiple resolutions are required. If dynamic shapes are essential, please open a new issue with the full text log, yolo checks output, and JetPack/TensorRT/Ultralytics versions so we can reproduce it; see the TensorRT export guide.

  5. Haotian GongHaotian Gong(edited)

    Hi, everyone,

    I'd like to flag two issues with the Jetson installation guide that may need updating.

    1. PyTorch 2.10.0 / Torchvision 0.25.0 doesn't work on JetPack 6.1

    Following the "Start with naive installation" and "Run on JetPack 6.1" sections, I installed Pytorch 2.10.0 and Torchvision 0.25.0 as instructed.

    Running my script then throws:

    ImportError: libcublas.so.13: cannot open shared object file: No such file or directory

    So I have to install Pytorch 2.8.0 with the guide:

    https://forums.developer.nvidia.com/t/jetson-orin-nano-issues-running-yolo-ultralytics-due-to-pytorch-torchvision-compatibility-jetpack-6-x-cuda-12-6/365683/9

    And this version of Pytorch works. So maybe is the guide on Jetson obsolete or mistaken?

    2. CSI camera requires the JetPack-bundled OpenCV, not the PyPI one

    On Jetson, using a CSI camera (via nvarguscamerasrc) requires the OpenCV that ships with JetPack, because the PyPI opencv-python wheel on aarch64 is built without GStreamer support.

    However, pip install ultralytics pulls in opencv-python as a dependency, which shadows the JetPack-bundled version and silently breaks CSI camera pipelines.

    pip uninstall -y opencv-python opencv-python-headless
sudo apt install -y python3-opencv   # JetPack-bundled OpenCV with GStreamer

    Thanks!

    1. Lakshantha DissanayakeLakshantha DissanayakeUltralytics

      Hi @ZephyrKeXiner Could you please double check whether you are using JetPack 6.1 system with CUDA 12.6? I have just verified on my JetPack 6.1 system with CUDA 12.6 with torch-2.10.0 and torchvision-0.25.0 as mentioned in the Ultralytics Jetson Guide and it works well.

      For JetPack 6.1, output of cat /etc/nv_tegra_release should look like below:

      (env) nvidia@ubuntu:~/yolo26/ultralytics$ cat /etc/nv_tegra_release 
# R36 (release), REVISION: 4.3, GCID: 38968081, BOARD: generic, EABI: aarch64, DATE: Wed Jan  8 01:49:37 UTC 2025
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

      Could you also share yours please? Thanks!

      cc: @raimbekovm

      1
    2. Haotian GongHaotian Gong

      Hi @lakshanthad,

      My output of cat /etc/nv_tegra_release is below:

      $ cat /etc/nv_tegra_release 
# R36 (release), REVISION: 5.0, GCID: 43688277, BOARD: generic, EABI: aarch64, DATE: Fri Jan 16 03:50:45 UTC 2026
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

      I notice that the revision is 5.0 not 4.3. Maybe does it cause the error?

  6. smbunnsmbunnGitHub(edited)

    I would have thought the Jetson Orin Nano could do more than 5 streams, the deepstream examples with cars handles 16 streams of 1080p at what appears to be full framerate.

    As a control systems engineer by training, edge devices seem a much better way to go so I prefer your jetson design. Even if one Jetson has an issue you are still processing 45 streams whereas a problem on your centralised server in option 2 takes everything down.

    Are you going to set up an MQTT feed for alerts from each stream? Maybe consider secondary processing and a buffer, so the edge device detects what it thinks is an issue and a centralised server grabs 5 seconds of video of the incident and double checks.

    Simon BUNN

    General

    From: Yash Patel @.> Sent: Friday, 22 May 2026 4:00 am To: ultralytics/ultralytics @.> Cc: Bunn, Simon @.>; Mention @.> Subject: Re: [ultralytics/ultralytics] guides/nvidia-jetson/ (Discussion #9942)

    Hi, I am designing an AI-based CCTV surveillance system for a residential society with 50 RTSP camera streams. My current architecture uses multiple NVIDIA Jetson Orin Nano Super Developer Kit devices for distributed inference processing. I ZjQcmQRYFpfptBannerStart Attention The sender's identity could not be verified and someone may be impersonating the sender. L'identité de l'expéditeur n'a pas pu être vérifiée et quelqu'un pourrait se faire passer pour l'expéditeur.

    ZjQcmQRYFpfptBannerEnd

    Hi, I am designing an AI-based CCTV surveillance system for a residential society with 50 RTSP camera streams. My current architecture uses multiple NVIDIA Jetson Orin Nano Super Developer Kit devices for distributed inference processing. I estimate that one Jetson Orin Nano Super (8GB) can process around 5 RTSP streams at usable real-time FPS. This requires approximately 10 Jetson devices for the full deployment. Each device costs around ₹33,000 (~USD 395). The total estimated hardware cost is ₹330,000 (~USD 3,950). I want to compare this setup against a centralized Linux-based custom PC/server with NVIDIA GPU acceleration under the same budget. The comparison focuses on AI inference performance, scalability, reliability, and 24/7 operational efficiency. The workload mainly includes multi-stream YOLO/object detection inference on live RTSP feeds. I want to determine which architecture provides the best computation-to-cost ratio for this surveillance use case.

    — Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/orgs/ultralytics/discussions/9942*discussioncomment-17006943__;Iw!!ElGdukoduuk!URiPjKkgDoMT_k2Y6pEd3XMEBGMDjhwgtQmwWXR9F4R4L7ymNloZD79NanOjYVg9YYmaARqyZGbgFv7U15KzgdcU$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABAAMILYUP7PPODNHDR3HC3434RYLAVCNFSM6AAAAABGAK2IGGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMBQGY4TIMY__;!!ElGdukoduuk!URiPjKkgDoMT_k2Y6pEd3XMEBGMDjhwgtQmwWXR9F4R4L7ymNloZD79NanOjYVg9YYmaARqyZGbgFv7U1zjhi4YU$. Triage notifications on the go with GitHub Mobile for iOShttps://urldefense.com/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!ElGdukoduuk!URiPjKkgDoMT_k2Y6pEd3XMEBGMDjhwgtQmwWXR9F4R4L7ymNloZD79NanOjYVg9YYmaARqyZGbgFv7U13gOq1qb$ or Androidhttps://urldefense.com/v3/__https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!ElGdukoduuk!URiPjKkgDoMT_k2Y6pEd3XMEBGMDjhwgtQmwWXR9F4R4L7ymNloZD79NanOjYVg9YYmaARqyZGbgFv7U1zOl3XaF$. You are receiving this because you were mentioned.Message ID: @.***>

    1. Glenn JocherGlenn JocherUltralytics

      5 streams on an Orin Nano Super is just a conservative YOLO/Python estimate; with a headless TensorRT or DeepStream pipeline, MAX power mode, and jetson_clocks, higher 1080p stream counts are definitely possible and depend heavily on the model size, input resolution, decode/encode, and any tracking or alert logic you add, so please drop your exact model, imgsz, FPS target, and RTSP resolution here and we can help size it more accurately using the Jetson deployment guide and DeepStream Jetson guide.

  7. yashpatel2010yashpatel2010GitHub

    Hi, I am designing an AI-based CCTV surveillance system for a residential society with 50 RTSP camera streams. My current architecture uses multiple NVIDIA Jetson Orin Nano Super Developer Kit devices for distributed inference processing. I estimate that one Jetson Orin Nano Super (8GB) can process around 5 RTSP streams at usable real-time FPS. This requires approximately 10 Jetson devices for the full deployment. Each device costs around ₹33,000 (~USD 395). The total estimated hardware cost is ₹330,000 (~USD 3,950). I want to compare this setup against a centralized Linux-based custom PC/server with NVIDIA GPU acceleration under the same budget. The comparison focuses on AI inference performance, scalability, reliability, and 24/7 operational efficiency. The workload mainly includes multi-stream YOLO/object detection inference on live RTSP feeds. I want to determine which architecture provides the best computation-to-cost ratio for this surveillance use case.

  8. RotatingGiraffeRotatingGiraffeGitHub

    Hi there, I am trying to run a fine-tuned YOLO26 based model for an OBB task on a Xavier NX running JetPack 5. I am using your container (ultralytics/ultralytics:latest-jetson-jetpack5) and trying to compile from PyTorch to TensorRT. I get the error: "mymodel.pt requires numpy>=1.26.1, however numpy==1.23.5 is installed."

    Previously I have successfully the file on my host machine from PT to ONNX, copied ONNX to Jetson and then used tensorrtexec to compile to Engine. But the resulting models did not create any bounding boxes.

    I wanted to ask if JetPack 5 even supports Yolo26 or if the model contains some incompatible layers? YOLO11 OBB works fine which I find extra-weird. Or is there likely an error somewhere else in my script?

    1. Glenn JocherGlenn JocherUltralytics

      Thanks for posting this here—JetPack 5 on Xavier NX is supported for YOLO26/TensorRT in our Jetson guide, so the numpy>=1.26.1 error points more to an environment/checkpoint mismatch than unsupported OBB layers; please paste the full text traceback and yolo checks output, and note that an engine built with tensorrtexec can still produce empty OBB results if the Ultralytics export metadata/postprocessing is not preserved.

    2. RotatingGiraffeRotatingGiraffeGitHub

      Hi there, I tried to create a minimal reproducible example for you, based on what I am doing. I hope it helps with debugging.

      The yolo checks you asked for is in the script output, along with the export script. I've also included the bit where I export via trtexex just for the sake of completion - it might very well be possible that I omit required metadata with this. My prefered goal would be that everything works via the Ultralytics Python API, but I had to search for workaround due to the NumPy error when compiling PT files.

      I am now using the Ultralytics base image in the MRE which I hoped would be easier to debug for us. The NumPy error persists on the new image. Previously I was using something built upon l4t-ml.

      Additionally, ONNX fails with an error ( `GLIBCXX_3.4.29' not found I think it was) but that is not a concern to me since I engine would be my prefered way to go. I have not installed any additional packages and kept it very vanilla (see dockerfile). The MRE also weirdly produced a new error during engine inference, but I'd say lets just focus on getting the Ultralytics native pipeline to work and then maybe all the other issues were just from my workarounds and disappear.

      Execution Command

      sudo docker compose up --build

      Directory Structure

      obb-test-repro/
 ┣ 📂data/
 ┃ ┗ 📜mydataset.yaml
 ┣ 📂models/
 ┃ ┣ 📜my-yolo26m-model.onnx
 ┃ ┗ 📜my-yolo26m-model.pt
 ┣ 📂src/
 ┃ ┗ 📜main.py
 ┣ 📜docker-compose.yml
 ┣ 📜Dockerfile
 ┗ 📜requirements.txt

      Dockerfile

      FROM ultralytics/ultralytics:latest-jetson-jetpack5

WORKDIR /app

# Copy the test app
COPY src/ ./src/

# Run the test
CMD ["python3", "src/main.py"]

      docker-compose.yml

      version: '3.8'

services:
  obb-test-jp5:
    image: obb-test:latest
    container_name: obb_test_jp5
    build:
      context: .
      dockerfile: Dockerfile
    runtime: nvidia
    network_mode: host
    ipc: host
    restart: "no"
    volumes:
      - ./models:/app/models
      - ./data:/app/data
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=all

      main.py

      """
Minimal OBB model test script.

Behavior:
- Runs `yolo checks` to output environment details.
- Uses files in `/app/models` (container path mapped from `obb-test/models`).
- Expects two files (hard-coded):
  - `/app/models/my-yolo26m-model.pt` (optional) — used for Ultralytics native export
  - `/app/models/my-yolo26m-model.onnx` (optional) — used for trtexec compilation
- Runs both export methods (if files exist) and validates inference on each compiled engine using mydataset dataset.
"""

import logging
import sys
import subprocess
import os
from pathlib import Path
from ultralytics import YOLO

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)


def run_yolo_checks():
    """Execute yolo checks via CLI to log environment details."""
    logger.info("=" * 60)
    logger.info("Environment Verification (yolo checks)")
    logger.info("=" * 60)
    try:
        res = subprocess.run(["yolo", "checks"], capture_output=True, text=True, check=True)
        logger.info("\n" + res.stdout.strip())
    except subprocess.CalledProcessError as e:
        logger.error(f"yolo checks failed:\n{e.stderr}")
    except FileNotFoundError:
        logger.error("yolo CLI command not found. Ensure ultralytics is installed in this environment.")


def test_export_to_engine():
    models_dir = Path("/app/models")
    pt_path = models_dir / "my-yolo26m-model.pt"
    onnx_path = models_dir / "my-yolo26m-model.onnx"
    dataset_yaml = "/app/data/mydataset.yaml"

    logger.info("=" * 60)
    logger.info("Minimal OBB Test - Export & Validation")
    logger.info("=" * 60)

    engines_tested = []

    # Part 1: Ultralytics native export (PT -> Engine)
    if pt_path.exists():
        try:
            logger.info(f"\nRunning Ultralytics native export for: {pt_path.name}")
            model = YOLO(str(pt_path))
            exported = model.export(format="engine", imgsz=1024, device=0, half=True)
            
            # Resolve the exported engine path
            native_engine = str(exported) if isinstance(exported, (str, Path)) else str(pt_path.with_suffix('.engine'))
            
            if os.path.exists(native_engine):
                logger.info(f"Ultralytics export successfully produced: {native_engine}")
                engines_tested.append(native_engine)
            else:
                logger.error(f"Ultralytics export completed, but engine file was not found at {native_engine}")
        except Exception as e:
            logger.error(f"Ultralytics native export failed: {e}", exc_info=True)
    else:
        logger.info(f"PT file not found ({pt_path}); skipping Ultralytics native export.")

    # Part 2: trtexec compilation (ONNX -> Engine)
    if onnx_path.exists():
        try:
            logger.info(f"\nRunning trtexec to compile ONNX: {onnx_path.name}")
            trt_engine = str(onnx_path.with_suffix('.trt.engine'))
            cmd = [
                "/usr/src/tensorrt/bin/trtexec",
                f"--onnx={onnx_path}",
                f"--saveEngine={trt_engine}",
                "--fp16",
                "--memPoolSize=workspace:768", # Prevents Out of Memory
            ]
            logger.info(f"Running command: {' '.join(cmd)}")
            res = subprocess.run(cmd, capture_output=True, text=True)
            
            if res.returncode != 0:
                logger.error("trtexec compilation failed.")
                logger.error(f"trtexec stderr:\n{res.stderr}")
            else:
                if os.path.exists(trt_engine):
                    logger.info(f"trtexec successfully saved engine to {trt_engine}")
                    engines_tested.append(trt_engine)
                else:
                    logger.error(f"trtexec returned success, but engine file was not found at {trt_engine}")
        except Exception as e:
            logger.error(f"trtexec execution failed: {e}", exc_info=True)
    else:
        logger.info(f"ONNX file not found ({onnx_path}); skipping trtexec compilation.")

    if not engines_tested:
        logger.error("\nNo engines are available to test. Exiting validation phase.")
        return False

    # Part 3: Inference Validation
    validation_success = True
    for eng in engines_tested:
        try:
            logger.info(f"\nValidating inference with engine: {eng}")
            
            model = YOLO(eng, task="obb")
            metrics = model.val(data=dataset_yaml, imgsz=1024, batch=1, verbose=False)
            
            logger.info(f"Validation metrics for {eng}: mAP50-95={metrics.box.map:.4f}, mAP50={metrics.box.map50:.4f}")
        except Exception as e:
            logger.error(f"Validation failed on {eng}: {e}", exc_info=True)
            validation_success = False

    logger.info("\nExecution complete.")
    return validation_success


if __name__ == "__main__":
    run_yolo_checks()
    
    try:
        success = test_export_to_engine()
        logger.info("\n" + "=" * 60)
        if success:
            logger.info("✓ Test passed")
        else:
            logger.info("✗ Test failed")
        logger.info("=" * 60)
        sys.exit(0 if success else 1)
    except Exception as e:
        logger.error(f"Unhandled script exception: {e}", exc_info=True)
        sys.exit(1)

      Output

      obb_test_jp5  | Creating new Ultralytics Settings v0.0.6 file ✅ 
obb_test_jp5  | View Ultralytics Settings with 'yolo settings' or at '/root/.config/Ultralytics/settings.json'
obb_test_jp5  | Update Settings with 'yolo settings key=value', i.e. 'yolo settings runs_dir=path/to/dir'. For help see https://docs.ultralytics.com/quickstart/#ultralytics-settings.
obb_test_jp5  | 2026-05-14 12:51:35,392 - INFO - ============================================================
obb_test_jp5  | 2026-05-14 12:51:35,392 - INFO - Environment Verification (yolo checks)
obb_test_jp5  | 2026-05-14 12:51:35,392 - INFO - ============================================================
obb_test_jp5  | 2026-05-14 12:51:50,554 - INFO - 
obb_test_jp5  | Ultralytics 8.4.50 🚀 Python-3.8.10 torch-2.1.0a0+41361538.nv23.06 CUDA:0 (Xavier, 6833MiB)
obb_test_jp5  | Setup complete ✅ (2 CPUs, 6.7 GB RAM, 91.7/107.6 GB disk)
obb_test_jp5  | 
obb_test_jp5  | OS                     Linux-5.10.216-tegra-aarch64-with-glibc2.29
obb_test_jp5  | Environment            Docker
obb_test_jp5  | Python                 3.8.10
obb_test_jp5  | Install                git
obb_test_jp5  | Path                   /ultralytics/ultralytics
obb_test_jp5  | RAM                    6.67 GB
obb_test_jp5  | Disk                   91.7/107.6 GB
obb_test_jp5  | CPU                    ARMv8 Processor rev 0 (v8l)
obb_test_jp5  | CPU count              2
obb_test_jp5  | GPU                    Xavier, 6833MiB
obb_test_jp5  | GPU count              1
obb_test_jp5  | CUDA                   11.4
obb_test_jp5  | 
obb_test_jp5  | numpy                  ✅ 1.23.5>=1.23.0
obb_test_jp5  | matplotlib             ✅ 3.7.5>=3.3.0
obb_test_jp5  | opencv-python-headless ✅ 4.13.0.92>=4.6.0
obb_test_jp5  | pillow                 ✅ 10.4.0>=7.1.2
obb_test_jp5  | pyyaml                 ✅ 6.0.3>=5.3.1
obb_test_jp5  | requests               ✅ 2.32.4>=2.23.0
obb_test_jp5  | scipy                  ✅ 1.10.1>=1.4.1
obb_test_jp5  | torch                  ✅ 2.1.0a0+41361538.nv23.6>=1.8.0
obb_test_jp5  | torchvision            ✅ 0.16.2+c6f3977>=0.9.0
obb_test_jp5  | psutil                 ✅ 7.2.2>=5.8.0
obb_test_jp5  | polars                 ✅ 1.8.2>=0.20.0
obb_test_jp5  | ultralytics-thop       ✅ 2.0.19>=2.0.18
obb_test_jp5  | torch                  ✅ 2.1.0a0+41361538.nv23.6!=2.4.0,>=1.8.0; sys_platform == "win32"
obb_test_jp5  | 2026-05-14 12:51:50,562 - INFO - ============================================================
obb_test_jp5  | 2026-05-14 12:51:50,563 - INFO - Minimal OBB Test - Export & Validation
obb_test_jp5  | 2026-05-14 12:51:50,564 - INFO - ============================================================
obb_test_jp5  | 2026-05-14 12:51:50,565 - INFO - 
obb_test_jp5  | Running Ultralytics native export for: my-yolo26m-model.pt
obb_test_jp5  | 2026-05-14 12:51:51,353 - ERROR - Ultralytics native export failed: ERROR ❌️ /app/models/my-yolo26m-model.pt requires numpy>=1.26.1, however numpy==1.23.5 is installed.
obb_test_jp5  | Traceback (most recent call last):
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/tasks.py", line 1466, in torch_safe_load
obb_test_jp5  |     ckpt = _load()
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/tasks.py", line 1463, in _load
obb_test_jp5  |     return torch_load(file, map_location="cpu")
obb_test_jp5  |   File "/ultralytics/ultralytics/utils/patches.py", line 181, in torch_load
obb_test_jp5  |     return torch.load(*args, **kwargs)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 877, in load
obb_test_jp5  |     return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 1253, in _load
obb_test_jp5  |     result = unpickler.load()
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 1246, in find_class
obb_test_jp5  |     return super().find_class(mod_name, name)
obb_test_jp5  | ModuleNotFoundError: No module named 'numpy._core'
obb_test_jp5  | 
obb_test_jp5  | The above exception was the direct cause of the following exception:
obb_test_jp5  | 
obb_test_jp5  | Traceback (most recent call last):
obb_test_jp5  |   File "src/minimal.py", line 58, in test_export_to_engine
obb_test_jp5  |     model = YOLO(str(pt_path))
obb_test_jp5  |   File "/ultralytics/ultralytics/models/yolo/model.py", line 76, in __init__
obb_test_jp5  |     super().__init__(model=model, task=task, verbose=verbose)
obb_test_jp5  |   File "/ultralytics/ultralytics/engine/model.py", line 144, in __init__
obb_test_jp5  |     self._load(model, task=task)
obb_test_jp5  |   File "/ultralytics/ultralytics/engine/model.py", line 283, in _load
obb_test_jp5  |     self.model, self.ckpt = load_checkpoint(weights)
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/tasks.py", line 1537, in load_checkpoint
obb_test_jp5  |     ckpt, weight = torch_safe_load(weight)  # load ckpt
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/tasks.py", line 1489, in torch_safe_load
obb_test_jp5  |     raise ModuleNotFoundError(
obb_test_jp5  | ModuleNotFoundError: ERROR ❌️ /app/models/my-yolo26m-model.pt requires numpy>=1.26.1, however numpy==1.23.5 is installed.
obb_test_jp5  | 2026-05-14 12:51:51,382 - INFO - 
obb_test_jp5  | Running trtexec to compile ONNX: my-yolo26m-model.onnx
obb_test_jp5  | 2026-05-14 12:51:51,382 - INFO - Running command: /usr/src/tensorrt/bin/trtexec --onnx=/app/models/my-yolo26m-model.onnx --saveEngine=/app/models/my-yolo26m-model.trt.engine --fp16 --memPoolSize=workspace:768
obb_test_jp5  | 2026-05-14 13:21:39,214 - INFO - trtexec successfully saved engine to /app/models/my-yolo26m-model.trt.engine
obb_test_jp5  | 2026-05-14 13:21:39,217 - INFO - 
obb_test_jp5  | Validating inference with engine: /app/models/my-yolo26m-model.trt.engine
obb_test_jp5  | Ultralytics 8.4.50 🚀 Python-3.8.10 torch-2.1.0a0+41361538.nv23.06 CUDA:0 (Xavier, 6833MiB)
obb_test_jp5  | Loading /app/models/my-yolo26m-model.trt.engine for TensorRT inference...
obb_test_jp5  | [05/14/2026-13:21:41] [TRT] [I] Loaded engine size: 43 MiB
obb_test_jp5  | [05/14/2026-13:21:45] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +260, GPU +306, now: CPU 648, GPU 5277 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +82, GPU +156, now: CPU 730, GPU 5433 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +41, now: CPU 0, GPU 41 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +28, now: CPU 687, GPU 5422 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +32, now: CPU 687, GPU 5454 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +92, now: CPU 0, GPU 133 (MiB)
obb_test_jp5  | Setting batch=1 input of shape (1, 3, 1024, 1024)
obb_test_jp5  | 2026-05-14 13:21:54,562 - INFO - generated new fontManager
obb_test_jp5  | val: Fast image access ✅ (ping: 1.1±0.9 ms, read: 479.2±146.1 MB/s, size: 2563.4 KB)
val: Scanning /app/data/labels/rec26.cache... 2940 images, 26 backgrounds, 0 corrupt: 100% ━━━━━━━━━━━━ 2940/2940 14.4Mit/s 0.0s
obb_test_jp5  | 2026-05-14 13:22:01,033 - ERROR - Validation failed on /app/models/my-yolo26m-model.trt.engine: CUDA error: no kernel image is available for execution on the device
obb_test_jp5  | CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
obb_test_jp5  | For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
obb_test_jp5  | Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
obb_test_jp5  | Traceback (most recent call last):
obb_test_jp5  |   File "src/minimal.py", line 114, in test_export_to_engine
obb_test_jp5  |     metrics = model.val(data=dataset_yaml, imgsz=1024, batch=1, verbose=False)
obb_test_jp5  |   File "/ultralytics/ultralytics/engine/model.py", line 615, in val
obb_test_jp5  |     validator(model=self.model)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
obb_test_jp5  |     return func(*args, **kwargs)
obb_test_jp5  |   File "/ultralytics/ultralytics/engine/validator.py", line 210, in __call__
obb_test_jp5  |     model.warmup(imgsz=(1 if pt else self.args.batch, self.data["channels"], imgsz, imgsz))  # warmup
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/autobackend.py", line 313, in warmup
obb_test_jp5  |     non_max_suppression(warmup_boxes)  # warmup NMS
obb_test_jp5  |   File "/ultralytics/ultralytics/utils/nms.py", line 154, in non_max_suppression
obb_test_jp5  |     i = torchvision.ops.nms(boxes, scores, iou_thres)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torchvision/ops/boxes.py", line 41, in nms
obb_test_jp5  |     return torch.ops.torchvision.nms(boxes, scores, iou_threshold)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/_ops.py", line 646, in __call__
obb_test_jp5  |     return self._op(*args, **kwargs or {})
obb_test_jp5  | RuntimeError: CUDA error: no kernel image is available for execution on the device
obb_test_jp5  | CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
obb_test_jp5  | For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
obb_test_jp5  | Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
obb_test_jp5  | 
obb_test_jp5  | 2026-05-14 13:22:01,521 - INFO - 
obb_test_jp5  | Execution complete.
obb_test_jp5  | 2026-05-14 13:22:01,524 - INFO - 
obb_test_jp5  | ============================================================
obb_test_jp5  | 2026-05-14 13:22:01,526 - INFO - ✗ Test failed
obb_test_jp5  | 2026-05-14 13:22:01,529 - INFO - ============================================================
obb_test_jp5 exited with code 1

      Export Device

      This is a bit of info on where my ONNX files come from and what my PC environment looks like where I compiled them.

      Yolo Checks on the ONNX Export Device

      Ultralytics 8.4.48 🚀 Python-3.13.9 torch-2.11.0+cu130 CPU (AMD Ryzen 5 5600G with Radeon Graphics)
Setup complete ✅ (12 CPUs, 15.0 GB RAM, 744.1/878.1 GB disk)

OS                     Linux-6.17.0-23-generic-x86_64-with-glibc2.39
Environment            Linux
Python                 3.13.9
Install                git
Path                   /home/myuser/projects/myproject/.venv/lib/python3.13/site-packages/ultralytics
RAM                    15.01 GB
Disk                   744.1/878.1 GB
CPU                    AMD Ryzen 5 5600G with Radeon Graphics
CPU count              12
GPU                    None
GPU count              None
CUDA                   None

numpy                  ✅ 2.4.4>=1.23.0
matplotlib             ✅ 3.10.9>=3.3.0
opencv-python          ✅ 4.13.0.92>=4.6.0
pillow                 ✅ 12.2.0>=7.1.2
pyyaml                 ✅ 6.0.3>=5.3.1
requests               ✅ 2.33.1>=2.23.0
scipy                  ✅ 1.17.1>=1.4.1
torch                  ✅ 2.11.0>=1.8.0
torch                  ✅ 2.11.0!=2.4.0,>=1.8.0; sys_platform == "win32"
torchvision            ✅ 0.26.0>=0.9.0
psutil                 ✅ 7.2.2>=5.8.0
polars                 ✅ 1.40.1>=0.20.0
ultralytics-thop       ✅ 2.0.19>=2.0.18

      Export Code

      from ultralytics import YOLO

# 1. Load the model weights
model_path = "my-yolo26m-model.pt"
model = YOLO(model_path)

# 2. Export the model
# Configured for OBB (Oriented Bounding Boxes) and legacy TensorRT compatibility
success = model.export(
    format="onnx", 
    imgsz=1024,      # Target resolution
    opset=13,       # ONNX operator set version, also tried 12
    simplify=True,   # I hoped this would help
    task="obb",
    half=False,      # I tried this with True and False
    end2end=False    # For legacy TensorRT support...?
)

print(f"Export successful: {success}")
      YOLO26m-obb summary (fused): 175 layers, 23,443,236 parameters, 0 gradients, 71.5 GFLOPs

PyTorch: starting from 'my-yolo26m-model.pt' with input shape (1, 3, 1024, 1024) BCHW and output shape(s) (1, 6, 21504) (90.8 MB)

ONNX: starting export with onnx 1.21.0 opset 13...
ONNX: slimming with onnxslim 0.1.93...
ONNX: export success ✅ 2.5s, saved as 'my-yolo26m-model.onnx' (81.3 MB)

Export complete (4.3s)

      Old Container

      Jut in case it is relevant, this is the yolo checks of my self-made container which is based on l4t-ml.

      root@xaviernx-desktop:/app# yolo checks
WARNING ⚠️ user config directory '/root/.config/Ultralytics' is not writable, using '/tmp/Ultralytics'. Set YOLO_CONFIG_DIR to override.
Creating new Ultralytics Settings v0.0.6 file ✅ 
View Ultralytics Settings with 'yolo settings' or at '/tmp/Ultralytics/settings.json'
Update Settings with 'yolo settings key=value', i.e. 'yolo settings runs_dir=path/to/dir'. For help see https://docs.ultralytics.com/quickstart/#ultralytics-settings.
Ultralytics 8.4.49 🚀 Python-3.8.10 torch-2.0.0+nv23.05 CUDA:0 (Xavier, 6833MiB)
Setup complete ✅ (2 CPUs, 6.7 GB RAM, 84.9/107.6 GB disk)

OS                     Linux-5.10.216-tegra-aarch64-with-glibc2.29
Environment            Docker
Python                 3.8.10
Install                pip
Path                   /usr/local/lib/python3.8/dist-packages/ultralytics
RAM                    6.67 GB
Disk                   84.9/107.6 GB
CPU                    ARMv8 Processor rev 0 (v8l)
CPU count              2
GPU                    Xavier, 6833MiB
GPU count              1
CUDA                   11.4

numpy                  ✅ 1.23.5>=1.23.0
matplotlib             ✅ 3.7.5>=3.3.0
opencv-python          ❌ (not installed)>=4.6.0
pillow                 ✅ 10.2.0>=7.1.2
pyyaml                 ✅ 5.3.1>=5.3.1
requests               ✅ 2.32.4>=2.23.0
scipy                  ✅ 1.10.1>=1.4.1
torch                  ✅ 2.0.0+nv23.5>=1.8.0
torch                  ✅ 2.0.0+nv23.5!=2.4.0,>=1.8.0; sys_platform == "win32"
torchvision            ✅ 0.15.1a0+42759b1>=0.9.0
psutil                 ✅ 5.9.6>=5.8.0
polars                 ❌ (not installed)>=0.20.0
ultralytics-thop       ❌ (not installed)>=2.0.18
    3. Murat RaimbekovMurat Raimbekov

      Thanks for the detailed MRE. I reproduced the numpy failure end-to-end with torch.save/torch.load and have a verified fix. There are two independent issues here.

      1. numpy._core / "requires numpy>=1.26.1" — root cause and fix

      Your .pt was saved on your PC with numpy 2.x / Python 3.13, so the checkpoint pickle embeds the numpy-2 numpy._core module path. JetPack 5 on Xavier NX runs Python 3.8, whose maximum numpy is 1.24.x, and Xavier cannot move to JetPack 6 (Orin-only). I verified that even numpy 1.26.x does not fully read such a checkpoint, so "upgrade numpy" is not a real option on JP5.

      The reliable fix is a small compatibility shim that aliases numpy._core to the legacy numpy.core, injected before the model is loaded. I confirmed this loads a numpy-2 torch.save checkpoint correctly under numpy 1.23.5 with the model and all fields intact. Put this at the very top of your main.py, before from ultralytics import YOLO:

      import sys, numpy, numpy.core
sys.modules["numpy._core"] = numpy.core
for _s in ("multiarray", "numeric", "umath", "_multiarray_umath"):
    try:
        sys.modules[f"numpy._core.{_s}"] = __import__(f"numpy.core.{_s}", fromlist=[_s])
    except Exception:
        pass

      Alternatively, fix the checkpoint once on the PC (where numpy 2.x can read it) so the device never needs the shim — load it and re-save it after converting the auxiliary numpy fields, then copy the new .pt to the Jetson.

      2. Empty boxes / CUDA error: no kernel image is available for execution on the device

      These are separate from numpy:

      • The trtexec-built engine produces no detections because a raw trtexec engine carries none of the Ultralytics export metadata, so the OBB decoding/post-processing is never applied. Build the engine on the Jetson with Ultralytics directly from the .pt (after applying the shim above) so metadata is embedded: YOLO("model.pt").export(format="engine", imgsz=1024, half=True, device=0).
      • no kernel image is available is raised inside torchvision.ops.nms on the GPU: the PyTorch/torchvision in ultralytics/ultralytics:latest-jetson-jetpack5 is built for Orin (SM 8.7), while Xavier NX is SM 7.2. That is exactly why YOLO11 OBB works in your older l4t-ml-based container (torch 2.0.0+nv23.5) but the official image fails. Recommended path: stay on the container that already works on your Xavier, pip install ultralytics opencv-python-headless there, and run the Ultralytics export(format="engine") + val() entirely on-device.

      TL;DR: the numpy error is a pickle module-path mismatch from the numpy-2 export PC (fix with the shim above — verified), and the empty/CUDA errors are a raw-trtexec metadata loss plus a torch/torchvision build that doesn't target Xavier's SM 7.2.

      1
    4. RotatingGiraffeRotatingGiraffeGitHub

      Hi there, thank you for the help! I just recently got around to trying out the shim and now everything works just fine, thank you for your help!

  9. yashpatel2010yashpatel2010GitHub

    How to run deepstream for YOLO26 models in my jetson orin nano developer kit?

    Is there any documentation for deepstream on jetson? I don't want any installation errors of version missmatch.

    So give me any perfect guide link so that I will not miss anything while installation and run inference on live RTSP streams

    2. Glenn JocherGlenn JocherUltralytics

      Adding one detail for Jetson Orin Nano: keep the versions paired as JetPack 6.1 + DeepStream 7.1 from the Jetson quick-start and NVIDIA DeepStream getting started; for live RTSP, set the uri in deepstream_app_config.txt to your stream, and if you hit any mismatch please post the full text traceback plus yolo checks output here so the team can help.

    3. yashpatel2010yashpatel2010GitHub

      Thanks for your reply. Just onemore doubt to clear. docker pull nvcr.io/nvidia/deepstream:7.1-triton-multiarch

      if I use above docker container for my jetpack6.1 so it there is no issue of version missmatch of deepstream right?

      is it works very well with Yolo26 deepstream settings which is showing in this doc? https://docs.ultralytics.com/guides/deepstream-nvidia-jetson/

      what is your recommendation to run Multiple RTSP streams with deepstream with docker containers or without docker containers manual installation?

    4. Glenn JocherGlenn JocherUltralytics

      Yes, nvcr.io/nvidia/deepstream:7.1-triton-multiarch is the correct DeepStream 7.1 Jetson container for JetPack 6.1, but it still depends on the host having the JetPack BSP and NVIDIA container runtime installed, so it lowers mismatch risk rather than removing it completely; for YOLO26 + RTSP, the DeepStream multi-stream section is the right reference, and I’d suggest native setup first since that’s the install path used in the guide, then move to Docker once your pipeline is stable. (docs.ultralytics.com)

    5. yashpatel2010yashpatel2010GitHub(edited)

      Thanks. For native setup Is it good to install deepstream 7.1 while installing jetpack6.1 only?

  10. githubwaltgithubwaltGitHub

    Installing torch 2.11.0 from the official website pytorch.org on an AGX Orin 64Gb (dev kit) with JP6.2.2 or installing torch 2.10.0 and torchvision 0.25.0 following this page, give complaints about not being compatible with the Compute Capability (CC) of my machine which has 8.7, whatever that means. Only installing the previous versions torch 2.8.0 and torchvision 0.23.0 do work.

    I understand that torch needs to be tailored specifically for the Jetson family and that is what this page is all about, so why do I have this issue if I follow the instructions given here? Is it because I am on Jetpack 6.2.2 and here it's 6.1?

    1. githubwaltgithubwaltGitHub

      I forgot the warning and error output when running a python test script:

      UserWarning: Found GPU0 Orin which is of compute capability (CC) 8.7.The following list shows the CCs this version of PyTorch was built for and the hardware CCs it supports:- 8.0 which supports hardware CC >=8.0,<9.0 except {8.7}- 9.0 which supports hardware CC >=9.0,<10.0

      torch.AcceleratorError CUDA error: no kernel image is available for execution on the device

    2. Glenn JocherGlenn JocherUltralytics

      Thanks, that traceback helps — CC 8.7 is the AGX Orin GPU architecture, and no kernel image is available means the installed PyTorch wheel was built without sm_87 support. So yes, this is most likely a JetPack/PyTorch wheel mismatch: the native steps in the JetPack 6.1 section don’t necessarily map cleanly to JP6.2.2, even though the page includes JP6.2 benchmark coverage. For now I’d stick with the working torch 2.8.0 + torchvision 0.23.0 pair or use the JetPack 6 Docker setup; if you want us to check your exact environment, please post yolo checks and the full text traceback here in the discussion.

    4. Glenn JocherGlenn JocherUltralytics

      No worries — the duplicate is fine, and if you later confirm a stable JP6.2.2 + PyTorch pairing on AGX Orin, please add it here so we can reflect it in the Jetson guide.

    5. githubwaltgithubwaltGitHub

      Will do. Now trying to get the most out of the AGX. Experimenting with number of workers on a batch size of 16. So far 12 workers gave memory issues ... I chose 12 because of 12 cores, but that was a little too much. Now running with 8 workers ... so far so good. To be continued 🙂

    6. githubwaltgithubwaltGitHub

      Hi Glenn, I can confirm a stable JP6.2.2 + Pytorch (torch 2.8.0 and torchvision 0.23.0) on the AGX Orin.