Ultralytics YOLO27： 加入候补名单。
开始使用

快速入门指南：使用 Ultralytics YOLO26 和 NVIDIA Jetson

本指南将带你了解如何在 NVIDIA Jetson 设备上部署 Ultralytics YOLO26，并提供性能基准测试，展示 YOLO26 在这些小巧节能的设备上的表现。

新增产品支持

我们已将本指南更新为最新的 NVIDIA Jetson AGX Thor Developer Kit：它可提供高达 2070 FP4 TFLOPS 的 AI 算力和 128 GB 内存，功耗可配置为 40 W 至 130 W。与 NVIDIA Jetson AGX Orin 相比，它的 AI 算力提升了 7.5 倍以上，运行最流行的 AI 模型时能效提升了 3.5 倍。



观看：使用 Ultralytics YOLO26 在 NVIDIA Jetson 设备上实现更快推理 🚀
NVIDIA Jetson Ecosystem
注意

本指南已在搭载最新稳定版 JetPack 7.2 的 NVIDIA Jetson AGX Thor Developer Kit (Jetson T5000) 和 NVIDIA Jetson AGX Orin Developer Kit (64GB) 上进行测试；在运行 JP6.1 版 JetPack 的 NVIDIA Jetson Orin Nano Super Developer Kit 上进行测试；在基于 NVIDIA Jetson Orin NX 16GB、运行 JP6.0/JP5.1.3 版 JetPack 的 Seeed Studio reComputer J4012 上进行测试；以及在基于 NVIDIA Jetson Nano 4GB、运行 JP4.6.1 版 JetPack 的 Seeed Studio reComputer J1020 v2 上进行测试。预计本指南适用于整个 NVIDIA Jetson 硬件产品系列，包括最新设备和旧款设备。

什么是 NVIDIA Jetson？#

NVIDIA Jetson 是一系列嵌入式计算板卡，旨在为边缘设备提供加速 AI（人工智能）计算能力。这些小巧而强大的设备基于 NVIDIA GPU 架构构建，无需依赖云计算资源，即可直接在设备上运行复杂的 AI 算法和深度学习模型。Jetson 板卡常用于机器人、自动驾驶汽车、工业自动化，以及其他需要在本地以低延迟和高效率执行 AI 推理的应用场景。此外，这些板卡基于 ARM64 架构，与传统 GPU 计算设备相比功耗更低。

NVIDIA Jetson 系列对比#

NVIDIA Jetson AGX Thor 是 NVIDIA Jetson 系列的最新一代产品，基于 NVIDIA Blackwell 架构，其 AI 性能远超前几代产品。下表对比了生态系统中的几款 Jetson 设备。

Jetson AGX Thor (T5000)Jetson AGX Orin 64GBJetson Orin NX 16GBJetson Orin Nano SuperJetson AGX XavierJetson Xavier NXJetson Nano
AI 性能2070 TFLOPS275 TOPS100 TOPS67 TOPS32 TOPS21 TOPS472 GFLOPS
GPU配备 96 个 Tensor Core 的 2560 核 NVIDIA Blackwell 架构 GPU配备 64 个 Tensor Core 的 2048 核 NVIDIA Ampere 架构 GPU配备 32 个 Tensor Core 的 1024 核 NVIDIA Ampere 架构 GPU配备 32 个 Tensor Core 的 1024 核 NVIDIA Ampere 架构 GPU配备 64 个 Tensor Core 的 512 核 NVIDIA Volta 架构 GPU配备 48 个 Tensor Core 的 384 核 NVIDIA Volta™ 架构 GPU128 核 NVIDIA Maxwell™ 架构 GPU
GPU 最高频率1.57 GHz1.3 GHz918 MHz1020 MHz1377 MHz1100 MHz921MHz
CPU14 核 Arm® Neoverse®-V3AE 64 位 CPU，1MB L2 + 16MB L312 核 NVIDIA Arm® Cortex A78AE v8.2 64 位 CPU，3MB L2 + 6MB L38 核 NVIDIA Arm® Cortex A78AE v8.2 64 位 CPU，2MB L2 + 4MB L36 核 Arm® Cortex®-A78AE v8.2 64 位 CPU，1.5MB L2 + 4MB L38 核 NVIDIA Carmel Arm®v8.2 64 位 CPU，8MB L2 + 4MB L36 核 NVIDIA Carmel Arm®v8.2 64 位 CPU，6MB L2 + 4MB L3四核 Arm® Cortex®-A57 MPCore 处理器
CPU 最高频率2.6 GHz2.2 GHz2.0 GHz1.7 GHz2.2 GHz1.9 GHz1.43GHz
内存128GB 256 位 LPDDR5X，273GB/s64GB 256 位 LPDDR5，204.8GB/s16GB 128 位 LPDDR5，102.4GB/s8GB 128 位 LPDDR5，102 GB/s32GB 256 位 LPDDR4x，136.5GB/s8GB 128 位 LPDDR4x，59.7GB/s4GB 64 位 LPDDR4，25.6GB/s

什么是 NVIDIA JetPack？#

NVIDIA JetPack SDK 为 Jetson 模块提供支持，并提供完整的开发环境，用于构建端到端加速 AI 应用。JetPack 包含 Jetson Linux（含引导加载程序、Linux 内核、Ubuntu 桌面环境），以及一整套用于加速 GPU 计算、多媒体、图形和计算机视觉的库。它还为主机和开发套件提供示例、文档及开发者工具，并支持更高级的 SDK，例如用于流式视频分析的 DeepStream、用于机器人的 Isaac，以及用于对话式 AI 的 Riva。

将 JetPack 刷写到 NVIDIA Jetson#

拿到 NVIDIA Jetson 设备后，第一步是将 NVIDIA JetPack 刷写到设备上。刷写 NVIDIA Jetson 设备有多种不同方式。

  1. 对于搭载 JetPack 7.2 的官方 Jetson AGX Thor、AGX Orin 或 Orin Nano Developer Kit，请下载统一版 Jetson ISO，将其写入 USB 闪存驱动器，然后按照对应设备的快速入门指南操作：AGX Thor、AGX Orin 或 Orin Nano。从 JetPack 7.2 开始，Orin Nano 不再使用可下载的 SD 卡映像；ISO USB 安装程序会将 Jetson Linux 安装到设备的 microSD 卡或 NVMe SSD 上。
  2. 如果你有意在 Jetson Orin Nano Developer Kit 上使用 JetPack 6，请按照 NVIDIA 的 JetPack 6.x 更新和 SD 卡说明操作。
  3. 如果你拥有其他 NVIDIA Development Kit，可以使用 SDK Manager 将 JetPack 刷写到设备上。
  4. 如果你拥有 Seeed Studio reComputer J4012 设备，可以将 JetPack 刷写到随附的 SSD；如果你拥有 Seeed Studio reComputer J1020 v2 设备，可以将 JetPack 刷写到 eMMC/ SSD。
  5. 如果你拥有其他由 NVIDIA Jetson 模块驱动的第三方设备，建议按照命令行刷写的方式操作。
注意

对于上述方法 1、4 和 5，在刷写系统并启动设备后，在设备终端中运行 sudo apt update && sudo apt install nvidia-jetpack -y，以安装所需的其余 JetPack 组件。

基于 Jetson 设备的 JetPack 支持情况#

下表列出了不同 NVIDIA Jetson 设备支持的 NVIDIA JetPack 版本。

JetPack 4JetPack 5JetPack 6JetPack 7
Jetson Nano✅❌❌❌
Jetson TX2✅❌❌❌
Jetson Xavier NX✅✅❌❌
Jetson AGX Xavier✅✅❌❌
Jetson AGX Orin❌✅✅✅
Jetson Orin NX❌✅✅✅
Jetson Orin Nano❌✅✅✅
Jetson AGX Thor❌❌❌✅

Docker 快速入门#

在 NVIDIA Jetson 上开始使用 Ultralytics YOLO26，最快的方法是运行预构建的 Jetson Docker 镜像。请参阅上表，并根据你使用的 Jetson 设备选择对应的 JetPack 版本。

t=ultralytics/ultralytics:latest-jetson-jetpack4
sudo docker pull $t && sudo docker run -it --ipc=host --runtime=nvidia $t
JetPack 7 Docker 和 TensorRT 支持

latest-nvidia-arm64 镜像使用 NVIDIA PyTorch 26.10，包括 CUDA 13.4 和 TensorRT 11.4。NVIDIA 在其兼容性表中列出了 PyTorch 对 JetPack 7.1 的支持，但 TensorRT 11 不支持 JetPack，包括 Thor。仅在受支持的主机上将此镜像用于 PyTorch 工作负载。对于 Jetson TensorRT 导出，请使用下方的原生安装方式，并搭配适用于你的 JetPack 版本且受支持的 TensorRT 10.x 运行时。在 Thor 或 Orin 上使用 JetPack 7.2 需要单独进行容器验证。请针对目标 GPU 和 TensorRT 版本重新构建引擎。

对于 JetPack 4、5 和 6 镜像，请继续参阅在 NVIDIA Jetson 上使用 TensorRT。上面的 JetPack 7.1 镜像仅适用于 PyTorch 工作负载。

从原生安装开始#

如需不使用 Docker 进行原生安装，请参阅以下步骤。

在 JetPack 7.2 上运行#

安装 Ultralytics 包#

在 Jetson 上安装 Ultralytics 软件包。首次导出模型时会自动安装导出所需的依赖，因此这里只需安装基础软件包。本指南重点介绍 NVIDIA TensorRT 导出，因为 TensorRT 在 Jetson 设备上性能最佳。

  1. 更新软件包列表、安装 pip 并升级到最新版本

    sudo apt update
sudo apt install python3-pip -y
pip install -U pip

  2. 安装 ultralytics pip 包

    pip install ultralytics

  3. 重启设备

    sudo reboot

安装 PyTorch 和 Torchvision#

上述 ultralytics 安装会安装 Torch 和 Torchvision。但通过 pip 安装的这两个软件包与运行 CUDA 13 的 JetPack 7.2 设备不兼容，因此需要手动安装。

根据 JP7.2 安装 torch 和 torchvision

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu130

安装 onnxruntime-gpu#

使用与你的 JetPack、Python 和 CUDA 版本匹配的 ONNX Runtime GPU wheel。目前 PyPI 发布版本包含 ARM64 wheel，但它们不能替代适用于所有 JetPack 版本的设备专用包。

下载并安装支持 Python3.12 的 onnxruntime-gpu 1.24.0。

pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/onnxruntime_gpu-1.24.0-cp312-cp312-linux_aarch64.whl

在 JetPack 6.1 上运行#

安装 Ultralytics 包#

在 Jetson 上安装 Ultralytics 软件包。首次导出模型时会自动安装导出所需的依赖，因此这里只需安装基础软件包。本指南重点介绍 NVIDIA TensorRT 导出，因为 TensorRT 在 Jetson 设备上性能最佳。

  1. 更新软件包列表、安装 pip 并升级到最新版本

    sudo apt update
sudo apt install python3-pip -y
pip install -U pip

  2. 安装 ultralytics pip 包

    pip install ultralytics

  3. 重启设备

    sudo reboot

安装 PyTorch 和 Torchvision#

上述 ultralytics 安装会安装 Torch 和 Torchvision。但通过 pip 安装的这两个软件包与基于 ARM64 架构的 Jetson 平台不兼容，因此你需要手动安装预构建的 PyTorch pip wheel，并从源代码编译或安装 Torchvision。

根据 JP6.1 安装 torch 2.10.0 和 torchvision 0.25.0

pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/torch-2.10.0-cp310-cp310-linux_aarch64.whl
pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/torchvision-0.25.0-cp310-cp310-linux_aarch64.whl
注意

访问 Jetson 版 PyTorch 页面，查看适用于不同 JetPack 版本的所有 PyTorch 版本。如需查看 PyTorch 和 Torchvision 兼容性详情，请访问 PyTorch 和 Torchvision 兼容性页面。

安装 cuDSS，以修复与 torch 2.10.0 相关的依赖问题

wget https://developer.download.nvidia.com/compute/cudss/0.7.1/local_installers/cudss-local-tegra-repo-ubuntu2204-0.7.1_0.7.1-1_arm64.deb
sudo dpkg -i cudss-local-tegra-repo-ubuntu2204-0.7.1_0.7.1-1_arm64.deb
sudo cp /var/cudss-local-tegra-repo-ubuntu2204-0.7.1/cudss-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cudss

安装 onnxruntime-gpu#

使用与你的 JetPack、Python 和 CUDA 版本匹配的 ONNX Runtime GPU wheel。目前 PyPI 发布版本包含 ARM64 wheel，但它们不能替代适用于所有 JetPack 版本的设备专用包。

你可以在 Jetson Zoo ONNX Runtime 兼容性矩阵中找到所有可用的 onnxruntime-gpu 包，其中按 JetPack 版本、Python 版本和其他兼容性详情进行了分类。

对于支持 Python 3.10 的 JetPack 6，你可以安装 onnxruntime-gpu 1.23.0：

pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/onnxruntime_gpu-1.23.0-cp310-cp310-linux_aarch64.whl

或者，对于 onnxruntime-gpu 1.20.0：

pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/onnxruntime_gpu-1.20.0-cp310-cp310-linux_aarch64.whl

在 JetPack 5.1.2 上运行#

安装 Ultralytics 包#

在 Jetson 上安装 Ultralytics 软件包。首次导出模型时会自动安装导出所需的依赖，因此这里只需安装基础软件包。本指南重点介绍 NVIDIA TensorRT 导出，因为 TensorRT 在 Jetson 设备上性能最佳。

  1. 更新软件包列表、安装 pip 并升级到最新版本

    sudo apt update
sudo apt install python3-pip -y
pip install -U pip

  2. 安装 ultralytics pip 包

    pip install ultralytics

  3. 重启设备

    sudo reboot

安装 PyTorch 和 Torchvision#

上述 ultralytics 安装会安装 Torch 和 Torchvision。但通过 pip 安装的这两个软件包与基于 ARM64 架构的 Jetson 平台不兼容，因此你需要手动安装预构建的 PyTorch pip wheel，并从源代码编译或安装 Torchvision。

  1. 卸载当前已安装的 PyTorch 和 Torchvision

    pip uninstall torch torchvision

  2. 根据 JP5.1.2 安装 torch 2.1.0 和 torchvision 0.16.2

    pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl
pip install https://github.com/ultralytics/assets/releases/download/v0.0.0/torchvision-0.16.2+c6f3977-cp38-cp38-linux_aarch64.whl
注意

访问 Jetson 版 PyTorch 页面，查看适用于不同 JetPack 版本的所有 PyTorch 版本。如需查看 PyTorch 和 Torchvision 兼容性详情，请访问 PyTorch 和 Torchvision 兼容性页面。

安装 onnxruntime-gpu#

使用与你的 JetPack、Python 和 CUDA 版本匹配的 ONNX Runtime GPU wheel。目前 PyPI 发布版本包含 ARM64 wheel，但它们不能替代适用于所有 JetPack 版本的设备专用包。

你可以在 Jetson Zoo ONNX Runtime 兼容性矩阵中找到所有可用的 onnxruntime-gpu 软件包，其中按 JetPack 版本、Python 版本及其他兼容性信息进行了分类。下载并安装支持 Python3.8 的 onnxruntime-gpu 1.17.0。

wget https://nvidia.box.com/shared/static/zostg6agm00fb6t5uisw51qi6kpcuwzd.whl -O onnxruntime_gpu-1.17.0-cp38-cp38-linux_aarch64.whl
pip install onnxruntime_gpu-1.17.0-cp38-cp38-linux_aarch64.whl
注意

onnxruntime-gpu 会自动将 NumPy 升级到最新版本，因此请重新安装 NumPy 1.23.5 来修复由此导致的问题：

pip install numpy==1.23.5

在 NVIDIA Jetson 上使用 TensorRT#

在 Ultralytics 支持的所有模型导出格式中，TensorRT 在 NVIDIA Jetson 设备上提供最高的推理性能，因此我们最推荐将其用于 Jetson 部署。导出之前，请安装适用于你的 JetPack 版本的 TensorRT Python 绑定，并通过 python3 -c "import tensorrt; print(tensorrt.__version__)" 验证。请在 JetPack 6/7 上使用受支持的 TensorRT 10.x 运行时；不要将其替换为 TensorRT 11。有关设置说明和高级用法，请参阅我们的专用 TensorRT 集成指南。

你也可以直接在浏览器中导出，无需在本地配置构建环境。在 Ultralytics Platform 模型导出选项卡中，选择 TensorRT 和目标 Jetson 设备。Thor 选项会在实体 Thor 硬件上进行验证。目前，六种 Orin 选项会生成以 AGX-Orin 为目标构建的候选引擎；部署前请在目标 Orin SKU 上进行验证。

TensorRT 引擎因设备而异

TensorRT 会在构建引擎的 GPU 上对引擎进行配置和优化。请匹配目标设备的 GPU 架构及 TensorRT/CUDA 运行时，并在部署设备上验证每个下载的引擎。Orin 设备即使架构相同，也不代表引擎一定可移植；如需获得最佳效果，INT8 校准应在目标设备上进行。

将模型转换为 TensorRT 并运行推理#

将 YOLO26n 的 PyTorch 格式模型转换为 TensorRT，然后使用导出的模型运行推理。

示例
from ultralytics import YOLO

# 加载 YOLO26n PyTorch 模型
model = YOLO("yolo26n.pt")

# 将模型导出为 TensorRT
model.export(format="engine")  # 创建 'yolo26n.engine'

# 加载导出的 TensorRT 模型
trt_model = YOLO("yolo26n.engine")

# 运行推理
results = trt_model("https://ultralytics.com/images/bus.jpg")

使用 NVIDIA 深度学习加速器 (DLA)#

NVIDIA 深度学习加速器 (DLA)是内置于 NVIDIA Jetson 设备中的专用硬件组件，可优化深度学习推理的能效和性能。DLA 将任务从 GPU 卸载出来（为更繁重的进程释放 GPU 资源），使模型能够以更低的功耗运行，同时保持较高的吞吐量，非常适合嵌入式系统和实时 AI 应用。

TensorRT 与 DLA 的兼容性

NVIDIA 将 TensorRT 10.7 列为最后一个支持 DLA 的版本；TensorRT 11.x 不支持 DLA。请使用你的 JetPack 版本支持且具备 DLA 能力的 TensorRT 版本。TensorRT 11 不支持 JetPack，包括仅使用 GPU 的导出。

以下 Jetson 设备配备了 DLA 硬件：

Jetson 设备DLA 核心数DLA 最高频率
Jetson AGX Orin 系列21.6 GHz
Jetson Orin NX 16GB2614 MHz
Jetson Orin NX 8GB1614 MHz
Jetson AGX Xavier 系列21.4 GHz
Jetson Xavier NX 系列21.1 GHz
示例
from ultralytics import YOLO

# 加载 YOLO26n PyTorch 模型
model = YOLO("yolo26n.pt")

# 启用 DLA 将模型导出为 TensorRT（仅适用于 FP16 或 INT8）
model.export(format="engine", device="dla:0", quantize=16)  # dla:0 或 dla:1 对应 DLA 核心

# 加载导出的 TensorRT 模型
trt_model = YOLO("yolo26n.engine")

# 运行推理
results = trt_model("https://ultralytics.com/images/bus.jpg")
注意

使用 DLA 导出时，某些层可能不受 DLA 支持，因此会回退到 GPU 执行。这种回退可能会增加延迟并影响整体推理性能。因此，与完全在 GPU 上运行 TensorRT 相比，DLA 的主要设计目标并非降低推理延迟，而是提高吞吐量和能效。

NVIDIA Jetson YOLO26 基准测试#

Ultralytics 团队针对 11 种不同模型格式进行了 YOLO26 基准测试，测量速度和精度：PyTorch、TorchScript、ONNX、OpenVINO、TensorRT、TF SavedModel、TF GraphDef、TF Lite、MNN、NCNN、ExecuTorch。测试使用 NVIDIA Jetson AGX Thor Developer Kit、NVIDIA Jetson AGX Orin Developer Kit（64GB）、NVIDIA Jetson Orin Nano Super Developer Kit，以及搭载 Jetson Orin NX 16GB 的 Seeed Studio reComputer J4012，在 FP32 精度下进行，默认输入图像尺寸为 640。

对比图表#

虽然所有模型导出格式都能在 NVIDIA Jetson 上运行，但下方的对比图仅包含 PyTorch、TorchScript 和 TensorRT。你可以在此图之后的部分查看所有导出格式的基准测试。

NVIDIA Jetson AGX Thor 开发者套件#

Jetson AGX Thor Benchmarks
基于 Ultralytics 8.3.226 进行基准测试

NVIDIA Jetson AGX Orin 开发者套件 (64GB)#

Jetson AGX Orin Benchmarks
基于 Ultralytics 8.4.32 进行基准测试

NVIDIA Jetson Orin Nano Super 开发者套件#

Jetson Orin Nano Super Benchmarks
基于 Ultralytics 8.4.33 进行基准测试

NVIDIA Jetson Orin NX 16GB#

Jetson Orin NX 16GB Benchmarks
基于 Ultralytics 8.4.33 进行基准测试

详细对比表#

下表展示了五种不同 YOLO26 模型（YOLO26n、YOLO26s、YOLO26m、YOLO26l、YOLO26x）在 11 种不同格式（PyTorch、TorchScript、ONNX、OpenVINO、TensorRT、TF SavedModel、TF GraphDef、TF Lite、MNN、NCNN、ExecuTorch）下的基准测试结果，包括每种组合的状态、大小、mAP50-95(B) 指标和推理时间。

NVIDIA Jetson AGX Thor 开发者套件#

性能
格式状态磁盘占用大小 (MB)mAP50-95(B)推理时间 (ms/im)
PyTorch✅5.30.47987.39
TorchScript✅9.80.47894.21
ONNX✅9.50.47676.58
OpenVINO✅10.10.479417.50
TensorRT (FP32)✅13.90.47911.90
TensorRT (FP16)✅7.60.47971.39
TensorRT (INT8)✅6.50.42731.52
TF SavedModel✅25.70.476447.24
TF GraphDef✅9.50.476445.98
TF Lite✅9.90.4764182.04
MNN✅9.40.478421.83

使用 Ultralytics 8.4.7 进行基准测试

注意

推理时间不包括预处理和后处理。

NVIDIA Jetson AGX Orin 开发者套件 (64GB)#

性能
格式状态磁盘占用大小 (MB)mAP50-95(B)推理时间 (ms/im)
PyTorch✅5.30.479011.58
TorchScript✅9.80.47704.60
ONNX✅9.50.47709.87
OpenVINO✅9.60.482028.80
TensorRT (FP32)✅11.50.47704.18
TensorRT (FP16)✅7.90.47892.62
TensorRT (INT8)✅5.40.46402.30
TF SavedModel✅24.60.476071.10
TF GraphDef✅9.50.476070.02
TF Lite✅9.90.4760227.94
MNN✅9.40.476032.46
NCNN✅9.30.481029.93

使用 Ultralytics 8.4.32 进行基准测试

注意

推理时间不包括预处理和后处理。

NVIDIA Jetson Orin Nano Super 开发者套件#

性能
格式状态磁盘占用大小 (MB)mAP50-95(B)推理时间 (ms/im)
PyTorch✅5.30.479015.60
TorchScript✅9.80.477012.60
ONNX✅9.50.476015.76
OpenVINO✅9.60.482056.23
TensorRT (FP32)✅11.30.47707.53
TensorRT (FP16)✅8.10.48004.57
TensorRT (INT8)✅5.30.44903.80
TF SavedModel✅24.60.4760118.33
TF GraphDef✅9.50.4760116.30
TF Lite✅9.90.4760286.00
MNN✅9.40.476068.77
NCNN✅9.30.481047.50

使用 Ultralytics 8.4.33 进行基准测试

注意

推理时间不包括预处理和后处理。

NVIDIA Jetson Orin NX 16GB#

性能
格式状态磁盘占用大小 (MB)mAP50-95(B)推理时间 (ms/im)
PyTorch✅5.30.479913.90
TorchScript✅9.80.478711.60
ONNX✅9.50.476314.18
OpenVINO✅9.60.481940.19
TensorRT (FP32)✅11.40.47707.01
TensorRT (FP16)✅8.00.47894.13
TensorRT (INT8)✅5.50.44893.49
TF SavedModel✅24.60.476492.34
TF GraphDef✅9.50.476492.06
TF Lite✅9.90.4764254.43
MNN✅9.40.476048.55
NCNN✅9.30.480534.31

使用 Ultralytics 8.4.33 进行基准测试

注意

推理时间不包括预处理和后处理。

了解 Seeed Studio 开展的更多基准测试，这些测试运行在不同版本的 NVIDIA Jetson 硬件上。

复现我们的结果#

如需在所有导出格式下复现上述 Ultralytics 基准测试，请运行以下代码：

示例
from ultralytics import YOLO

# 加载 YOLO26n PyTorch 模型
model = YOLO("yolo26n.pt")

# 在 COCO128 数据集上对 YOLO26n 的所有导出格式进行速度和精度基准测试
results = model.benchmark(data="coco128.yaml", imgsz=640)

请注意，基准测试结果可能因系统的具体硬件和软件配置，以及运行基准测试时系统的当前负载而有所不同。为获得最可靠的结果，请使用包含大量图像的数据集，例如 data='coco.yaml'（5000 张验证图像）。

使用 NVIDIA Jetson 的最佳实践#

遵循以下最佳实践，充分发挥 YOLO26 在 NVIDIA Jetson 上的性能。

  1. 启用 MAX 电源模式

    在 Jetson 上启用 MAX Power Mode，确保所有 CPU 和 GPU 核心均已开启。

    sudo nvpmodel -m 0

  2. 启用 Jetson Clocks

    启用 Jetson Clocks，确保所有 CPU 和 GPU 核心均以最高频率运行。

    sudo jetson_clocks

  3. 安装 Jetson Stats 应用

    使用 jetson-stats 应用监控系统组件的温度、查看 CPU、GPU 和 RAM 利用率、更改功耗模式、设置最高时钟频率，以及查看 JetPack 信息。

    sudo apt update
sudo pip install jetson-stats
sudo reboot
jtop
jtop dashboard from jetson-stats showing Jetson CPU, GPU, and memory utilization

NVIDIA Jetson 内存优化技巧#

可用内存通常是 Jetson 设备的性能瓶颈，尤其是在 Jetson Orin Nano (8 GB) 或 Orin NX 8 GB 等内存较小的型号上。以下技巧都是切实可行且风险较低的调整，综合运用可释放数百 MB 内存，让你运行更大的 YOLO 模型或支持更多并行工作负载。要了解全面的介绍，请参阅 NVIDIA 关于提升 Jetson 内存效率的博客。

1. 切换到无头（无图形界面）启动#

如果你的 Jetson 通过 SSH 连接，或作为未连接显示器的生产设备运行，移除桌面环境和显示服务器最多可释放 865 MB RAM：

sudo systemctl set-default multi-user.target
sudo reboot

稍后如需恢复桌面环境：

sudo systemctl set-default graphical.target
sudo reboot

2. 禁用未使用的系统服务#

非必要的后台服务（蓝牙、连接管理器、未使用的硬件守护进程）合计约占用 32 MB。列出当前运行的服务，并禁用部署不需要的服务：

# List running services
systemctl list-units --type=service --state=running

# Disable a service
sudo systemctl disable SERVICE_NAME

3. 分析内存使用情况#

开始优化前，先找出实际占用 RAM 的进程。procrank 会按 PSS（比例集大小）对进程排序。与 RSS（常驻集大小，即进程映射的物理 RAM 页面总量，包括与其他进程共享的页面）相比，PSS 能更准确地反映每个进程的实际内存占用：

git clone https://github.com/csimmonds/procrank_linux.git
cd procrank_linux && make
sudo ./procrank

查看每个进程的 GPU 和 NvMap（CUDA/视频管线）分配情况：

sudo cat /sys/kernel/debug/nvmap/iovmm/clients

4. 在生产环境中不连接显示器运行推理#

对于不需要实时预览的推理管线，禁用显示相关组件（Tiler、OSD、DisplaySink）仅从管线本身就能节省 200+ MB。使用 Ultralytics YOLO 时，关闭查看器并将结果写入磁盘：

示例
from ultralytics import YOLO

model = YOLO("yolo26n.engine")

# show=False 可阻止打开任何显示窗口；save=True 可将带标注的输出写入磁盘
results = model.predict(source="video.mp4", show=False, save=True)

累计影响#

优化项大约节省的内存
禁用桌面图形界面~865 MB
禁用未使用的操作系统服务~32 MB
无头推理管线（无显示）~200+ MB
合计（轻松实现的优化）~1 GB+

对于在内存受限设备上运行 TensorRT INT8 模型，这些改动组合起来尤其有价值——它们可能决定更大的模型变体能否装入内存。

后续步骤#

如需进一步学习和获取支持，请参阅 Ultralytics YOLO26 文档。

常见问题#

  • 要在 NVIDIA Jetson 设备上部署 Ultralytics YOLO26，请先使用 NVIDIA JetPack SDK 刷写 Jetson 设备。然后，你可以使用预构建的 Docker 镜像快速完成设置，或者手动安装所需软件包。

  • YOLO26 模型已在 NVIDIA Jetson AGX Thor、AGX Orin、Orin Nano Super 和 Orin NX 设备上进行基准测试，其中 TensorRT 格式的推理性能最佳。详细对比表部分列出了每种模型和导出格式的 mAP50-95 和推理时间。

  • 在 NVIDIA Jetson 上部署 YOLO26 模型时，建议使用 TensorRT，因为它会针对 Jetson GPU 优化模型，从而在所有导出格式中实现最快的推理速度。

  • 要在 NVIDIA Jetson 上安装 PyTorch 和 Torchvision，请先卸载可能通过 pip 安装的现有版本。然后，手动安装与 Jetson ARM64 架构兼容的 PyTorch 和 Torchvision 版本。有关此过程的详细说明，请参阅安装 PyTorch 和 Torchvision章节。

  • 要在 NVIDIA Jetson 上使用 YOLO26 获得最佳性能，请遵循以下最佳实践：

    1. 启用 MAX 电源模式，以使用所有 CPU 和 GPU 核心。
    2. 启用 Jetson Clocks，让所有核心以最高频率运行。
    3. 安装 Jetson Stats 应用以监控系统指标。

    有关命令和更多详情，请参阅使用 NVIDIA Jetson 的最佳实践章节。

  • 可用 RAM 通常是内存较小的 Jetson 设备的瓶颈。以下三项简单优化结合使用，可释放超过 1 GB 内存：

    1. 切换到无头启动（sudo systemctl set-default multi-user.target），移除桌面图形界面（可节省约 865 MB）。
    2. 禁用未使用的服务，例如蓝牙或连接管理器（可节省约 32 MB）。
    3. 不连接显示器运行推理：在 YOLO 的 predict 调用中设置 show=False，避免分配显示管线内存（可节省约 200+ MB）。

    使用 procrank 分析每个进程的 RAM 使用情况，并使用 sudo cat /sys/kernel/debug/nvmap/iovmm/clients 检查 GPU 分配。完整详情请参阅内存优化技巧章节。

  • 随 JetPack 6 发布的 TensorRT 10.3.0 存在一个已知问题，会导致无法使用 NMS-free（nms=False）或嵌入式 NMS（nms=True）输出构建 INT8 引擎。检测到这种组合时，Ultralytics 会为 NMS-free 导出选择一对多检测头，并因错误而停止 nms=True 导出。

    若要恢复这些 INT8 导出，请将 TensorRT 升级到较新版本（例如 10.7.0+）：

    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/arm64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get install -y tensorrt

    升级后，请重新运行导出。更多详情请参阅 GitHub issue #23841 和 issue #26655。

贡献者
glenn-jocherglenn-jocher33lakshanthadlakshanthad28RizwanMunawarRizwanMunawar5onuralpszronuralpszr4Laughing-qLaughing-q2Y-T-GY-T-G2raimbekovmraimbekovm2Burhan-QBurhan-Q2JWJohnny Wilson Dougherty1pderrengerpderrenger1picsalexpicsalex1justincdavisjustincdavis1MatthewNoyceMatthewNoyce1AhelsamahyAhelsamahy1

评论194

提出问题、分享结果，或帮助其他人。

  2. pknowlerGitHub

    Struggling with this on a new Jetson Orin Nano JP6.1. All good following the steps and installs to give torch.cuda.is_available as TRUE. The TensorRT export fails though with a number of dependency errors and I can’t re-install it with anything. Any pointers please?

    1
    1. Glenn JocherUltralytics

      @pknowler it seems like the dependency errors during TensorRT export might stem from compatibility issues. Please ensure you’ve installed the required JetPack-compatible versions of PyTorch, Torchvision, and TensorRT as outlined in the Jetson Quick Start Guide. Additionally, verify that onnxruntime-gpu and cuSPARSELt are correctly installed per the guide. If the issue persists, you can try reinstalling TensorRT using the NVIDIA SDK Manager for your JetPack version. Let me know if you need further clarification!

    2. pknowlerGitHub

      Thank you. I will try again working through all of the steps and let you know how it goes!

    3. pknowlerGitHub

      I’ve tried again but this as far as I get… I have followed everything carefully.

      WARNING ⚠️ TensorRT requires GPU export, automatically assigning device=0 Ultralytics 8.3.57 🚀 Python-3.10.16 torch-2.5.0a0+872d972e41.nv24.08 CUDA:0 (Orin, 7620MiB) YOLO11n summary (fused): 238 layers, 2,616,248 parameters, 0 gradients, 6.5 GFLOPs

      PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB)

      ONNX: starting export with onnx 1.17.0 opset 19... ONNX: slimming with onnxslim 0.1.46... ONNX: export success ✅ 3.1s, saved as 'yolo11n.onnx' (10.2 MB) requirements: Ultralytics requirement ['tensorrt>7.0.0,!=10.1.0'] not found, attempting AutoUpdate... error: subprocess-exited-with-error

      × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [6 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/tmp/pip-install-d0ljzl60/tensorrt-cu12_db0552ac431148489703f6f75df6764f/setup.py", line 71, in raise RuntimeError("TensorRT does not currently build wheels for Tegra systems") RuntimeError: TensorRT does not currently build wheels for Tegra systems [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

      × Encountered error while generating package metadata. ╰─> See above for output.

      note: This is an issue with the package mentioned above, not pip. hint: See above for details. Retry 1/2 failed: Command 'pip install --no-cache-dir "tensorrt>7.0.0,!=10.1.0" ' returned non-zero exit status 1. error: subprocess-exited-with-error

      × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [6 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "/tmp/pip-install-782zjqf9/tensorrt-cu12_44d261aea7c5489db3e9ce15cec8ee64/setup.py", line 71, in raise RuntimeError("TensorRT does not currently build wheels for Tegra systems") RuntimeError: TensorRT does not currently build wheels for Tegra systems [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

      × Encountered error while generating package metadata. ╰─> See above for output.

      note: This is an issue with the package mentioned above, not pip. hint: See above for details.

    5. Glenn JocherUltralytics

      @pknowler it seems your TensorRT version is not compatible with the Jetson's ARM64 architecture. TensorRT does not provide pre-built Python wheels for Tegra systems. Please ensure you have TensorRT installed via the NVIDIA SDK Manager as part of the JetPack installation, and confirm compatibility with your JetPack version. The Jetson Quick Start Guide might also help clarify setup specifics. Let me know if further assistance is needed!

    6. pknowlerGitHub

      @glenn-jocher thank you. I have found the issue now. I am using miniconda and for some reason TensorRT Python bindings are not available in conda envs. So I can import Torch with no problem but the import TensorRT fails. I have re-installed without conda and all works as expected! 👍🏻

    1. Paula DerrengerUltralytics(已编辑)

      In terms of reducing inference time on your NVIDIA Jetson device, using TensorRT is generally recommended. TensorRT is specifically designed to provide high performance for NVIDIA hardware by optimizing deep learning models for inference. It accelerates models by fusing layers, pruning unused layers, and quantizing variables, among other optimizations. 🚀

      DeepStream, on the other hand, is more focused on streaming analytics and can utilize TensorRT-optimized models. For most cases, directly using TensorRT for model optimization is your go-to for minimizing inference times.

      Here's a quick example of exporting a YOLO model to TensorRT and running inference, tailored for your convenience:

      from ultralytics import YOLO

# Load your YOLO model
model = YOLO('yolo26n.pt')

# Export to TensorRT format
model.export(format='engine')

# Load the TensorRT model
trt_model = YOLO('yolo26n.engine')

# Run inference using the TensorRT model
results = trt_model('your_image_or_video.jpg')

      Feel free to explore more about deploying on NVIDIA Jetson with our Quick Start Guide for NVIDIA Jetson. Happy inferencing! 🌟

      2
    3. Paula DerrengerUltralytics(已编辑)

      Hey there! 😊 It looks like you're interested in speeding up your inference times. Between TensorRT and DeepStream on NVIDIA Jetson devices, TensorRT typically offers more direct optimization for reducing inference time for individual models by providing lower-level optimizations and quantization options.

      DeepStream is a comprehensive framework for building end-to-end video analytics applications, which utilizes TensorRT under the hood for model inference. DeepStream excels in scenarios involving video streams, multi-stream input, and complex pipelines involving multiple AI models and analytics. If your primary goal is solely to minimize inference time for a specific model, direct optimization via TensorRT can provide more focused improvements.

      A quick example of exporting a YOLO model to TensorRT:

      from ultralytics import YOLO

# Load your YOLO model
model = YOLO('yolo26n.pt')

# Export to TensorRT format
model.export(format='engine')  # This creates 'yolo26n.engine'

# Done! Now you can use 'yolo26n.engine' for faster inference.

      However, if your application benefits from a broader video analytics pipeline, incorporating DeepStream into your workflow can optimize and streamline these processes on top of efficiently handling inference with TensorRT optimized models. 🚀

      Hope this gives you a clearer picture! If you have more questions, feel free to ask.

      1
  4. Yufeng Wu(已编辑)

    Hi, ultralytics team. I'm trying yolo26n-pose on Jetson orin agx 64gb and have this problem: I tried building engine with dynamic=true but failed, with the log saying ForeignNode[/model.23/Sigmoid.../model.23/Flatten] (Myelin) Skipping tactic ... Incompatible effective shapes in operation /model_23/Sub between: Unsqueeze_18_output_0 f32[1,2,33600] and Slice_output_0 f32[1,2,21504] Could not find any implementation for node {ForeignNode[/model.23/Sigmoid.../model.23/Flatten]} Is there a way to fix it?

    1. Glenn JocherUltralytics

      This indicates a TensorRT dynamic-shape profile failure in the pose head. As a workaround, export a fixed-resolution engine, e.g. yolo export model=yolo26n-pose.pt format=engine imgsz=640 batch=1 dynamic=False; build separate engines if multiple resolutions are required. If dynamic shapes are essential, please open a new issue with the full text log, yolo checks output, and JetPack/TensorRT/Ultralytics versions so we can reproduce it; see the TensorRT export guide.

  5. Haotian Gong(已编辑)

    Hi, everyone,

    I'd like to flag two issues with the Jetson installation guide that may need updating.

    1. PyTorch 2.10.0 / Torchvision 0.25.0 doesn't work on JetPack 6.1

    Following the "Start with naive installation" and "Run on JetPack 6.1" sections, I installed Pytorch 2.10.0 and Torchvision 0.25.0 as instructed.

    Running my script then throws:

    ImportError: libcublas.so.13: cannot open shared object file: No such file or directory

    So I have to install Pytorch 2.8.0 with the guide:

    https://forums.developer.nvidia.com/t/jetson-orin-nano-issues-running-yolo-ultralytics-due-to-pytorch-torchvision-compatibility-jetpack-6-x-cuda-12-6/365683/9

    And this version of Pytorch works. So maybe is the guide on Jetson obsolete or mistaken?

    2. CSI camera requires the JetPack-bundled OpenCV, not the PyPI one

    On Jetson, using a CSI camera (via nvarguscamerasrc) requires the OpenCV that ships with JetPack, because the PyPI opencv-python wheel on aarch64 is built without GStreamer support.

    However, pip install ultralytics pulls in opencv-python as a dependency, which shadows the JetPack-bundled version and silently breaks CSI camera pipelines.

    pip uninstall -y opencv-python opencv-python-headless
sudo apt install -y python3-opencv   # JetPack-bundled OpenCV with GStreamer

    Thanks!

    1. Lakshantha DissanayakeUltralytics

      Hi @ZephyrKeXiner Could you please double check whether you are using JetPack 6.1 system with CUDA 12.6? I have just verified on my JetPack 6.1 system with CUDA 12.6 with torch-2.10.0 and torchvision-0.25.0 as mentioned in the Ultralytics Jetson Guide and it works well.

      For JetPack 6.1, output of cat /etc/nv_tegra_release should look like below:

      (env) nvidia@ubuntu:~/yolo26/ultralytics$ cat /etc/nv_tegra_release 
# R36 (release), REVISION: 4.3, GCID: 38968081, BOARD: generic, EABI: aarch64, DATE: Wed Jan  8 01:49:37 UTC 2025
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

      Could you also share yours please? Thanks!

      cc: @raimbekovm

      1
    2. Haotian Gong

      Hi @lakshanthad,

      My output of cat /etc/nv_tegra_release is below:

      $ cat /etc/nv_tegra_release 
# R36 (release), REVISION: 5.0, GCID: 43688277, BOARD: generic, EABI: aarch64, DATE: Fri Jan 16 03:50:45 UTC 2026
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

      I notice that the revision is 5.0 not 4.3. Maybe does it cause the error?

  6. smbunnGitHub(已编辑)

    I would have thought the Jetson Orin Nano could do more than 5 streams, the deepstream examples with cars handles 16 streams of 1080p at what appears to be full framerate.

    As a control systems engineer by training, edge devices seem a much better way to go so I prefer your jetson design. Even if one Jetson has an issue you are still processing 45 streams whereas a problem on your centralised server in option 2 takes everything down.

    Are you going to set up an MQTT feed for alerts from each stream? Maybe consider secondary processing and a buffer, so the edge device detects what it thinks is an issue and a centralised server grabs 5 seconds of video of the incident and double checks.

    Simon BUNN

    General

    From: Yash Patel @.> Sent: Friday, 22 May 2026 4:00 am To: ultralytics/ultralytics @.> Cc: Bunn, Simon @.>; Mention @.> Subject: Re: [ultralytics/ultralytics] guides/nvidia-jetson/ (Discussion #9942)

    Hi, I am designing an AI-based CCTV surveillance system for a residential society with 50 RTSP camera streams. My current architecture uses multiple NVIDIA Jetson Orin Nano Super Developer Kit devices for distributed inference processing. I ZjQcmQRYFpfptBannerStart Attention The sender's identity could not be verified and someone may be impersonating the sender. L'identité de l'expéditeur n'a pas pu être vérifiée et quelqu'un pourrait se faire passer pour l'expéditeur.

    ZjQcmQRYFpfptBannerEnd

    Hi, I am designing an AI-based CCTV surveillance system for a residential society with 50 RTSP camera streams. My current architecture uses multiple NVIDIA Jetson Orin Nano Super Developer Kit devices for distributed inference processing. I estimate that one Jetson Orin Nano Super (8GB) can process around 5 RTSP streams at usable real-time FPS. This requires approximately 10 Jetson devices for the full deployment. Each device costs around ₹33,000 (~USD 395). The total estimated hardware cost is ₹330,000 (~USD 3,950). I want to compare this setup against a centralized Linux-based custom PC/server with NVIDIA GPU acceleration under the same budget. The comparison focuses on AI inference performance, scalability, reliability, and 24/7 operational efficiency. The workload mainly includes multi-stream YOLO/object detection inference on live RTSP feeds. I want to determine which architecture provides the best computation-to-cost ratio for this surveillance use case.

    — Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/orgs/ultralytics/discussions/9942*discussioncomment-17006943__;Iw!!ElGdukoduuk!URiPjKkgDoMT_k2Y6pEd3XMEBGMDjhwgtQmwWXR9F4R4L7ymNloZD79NanOjYVg9YYmaARqyZGbgFv7U15KzgdcU$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABAAMILYUP7PPODNHDR3HC3434RYLAVCNFSM6AAAAABGAK2IGGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMBQGY4TIMY__;!!ElGdukoduuk!URiPjKkgDoMT_k2Y6pEd3XMEBGMDjhwgtQmwWXR9F4R4L7ymNloZD79NanOjYVg9YYmaARqyZGbgFv7U1zjhi4YU$. Triage notifications on the go with GitHub Mobile for iOShttps://urldefense.com/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!ElGdukoduuk!URiPjKkgDoMT_k2Y6pEd3XMEBGMDjhwgtQmwWXR9F4R4L7ymNloZD79NanOjYVg9YYmaARqyZGbgFv7U13gOq1qb$ or Androidhttps://urldefense.com/v3/__https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!ElGdukoduuk!URiPjKkgDoMT_k2Y6pEd3XMEBGMDjhwgtQmwWXR9F4R4L7ymNloZD79NanOjYVg9YYmaARqyZGbgFv7U1zOl3XaF$. You are receiving this because you were mentioned.Message ID: @.***>

    1. Glenn JocherUltralytics

      5 streams on an Orin Nano Super is just a conservative YOLO/Python estimate; with a headless TensorRT or DeepStream pipeline, MAX power mode, and jetson_clocks, higher 1080p stream counts are definitely possible and depend heavily on the model size, input resolution, decode/encode, and any tracking or alert logic you add, so please drop your exact model, imgsz, FPS target, and RTSP resolution here and we can help size it more accurately using the Jetson deployment guide and DeepStream Jetson guide.

  7. yashpatel2010GitHub

    Hi, I am designing an AI-based CCTV surveillance system for a residential society with 50 RTSP camera streams. My current architecture uses multiple NVIDIA Jetson Orin Nano Super Developer Kit devices for distributed inference processing. I estimate that one Jetson Orin Nano Super (8GB) can process around 5 RTSP streams at usable real-time FPS. This requires approximately 10 Jetson devices for the full deployment. Each device costs around ₹33,000 (~USD 395). The total estimated hardware cost is ₹330,000 (~USD 3,950). I want to compare this setup against a centralized Linux-based custom PC/server with NVIDIA GPU acceleration under the same budget. The comparison focuses on AI inference performance, scalability, reliability, and 24/7 operational efficiency. The workload mainly includes multi-stream YOLO/object detection inference on live RTSP feeds. I want to determine which architecture provides the best computation-to-cost ratio for this surveillance use case.

  8. RotatingGiraffeGitHub

    Hi there, I am trying to run a fine-tuned YOLO26 based model for an OBB task on a Xavier NX running JetPack 5. I am using your container (ultralytics/ultralytics:latest-jetson-jetpack5) and trying to compile from PyTorch to TensorRT. I get the error: "mymodel.pt requires numpy>=1.26.1, however numpy==1.23.5 is installed."

    Previously I have successfully the file on my host machine from PT to ONNX, copied ONNX to Jetson and then used tensorrtexec to compile to Engine. But the resulting models did not create any bounding boxes.

    I wanted to ask if JetPack 5 even supports Yolo26 or if the model contains some incompatible layers? YOLO11 OBB works fine which I find extra-weird. Or is there likely an error somewhere else in my script?

    1. Glenn JocherUltralytics

      Thanks for posting this here—JetPack 5 on Xavier NX is supported for YOLO26/TensorRT in our Jetson guide, so the numpy>=1.26.1 error points more to an environment/checkpoint mismatch than unsupported OBB layers; please paste the full text traceback and yolo checks output, and note that an engine built with tensorrtexec can still produce empty OBB results if the Ultralytics export metadata/postprocessing is not preserved.

    2. RotatingGiraffeGitHub

      Hi there, I tried to create a minimal reproducible example for you, based on what I am doing. I hope it helps with debugging.

      The yolo checks you asked for is in the script output, along with the export script. I've also included the bit where I export via trtexex just for the sake of completion - it might very well be possible that I omit required metadata with this. My prefered goal would be that everything works via the Ultralytics Python API, but I had to search for workaround due to the NumPy error when compiling PT files.

      I am now using the Ultralytics base image in the MRE which I hoped would be easier to debug for us. The NumPy error persists on the new image. Previously I was using something built upon l4t-ml.

      Additionally, ONNX fails with an error ( `GLIBCXX_3.4.29' not found I think it was) but that is not a concern to me since I engine would be my prefered way to go. I have not installed any additional packages and kept it very vanilla (see dockerfile). The MRE also weirdly produced a new error during engine inference, but I'd say lets just focus on getting the Ultralytics native pipeline to work and then maybe all the other issues were just from my workarounds and disappear.

      Execution Command
      sudo docker compose up --build
      Directory Structure
      obb-test-repro/
 ┣ 📂data/
 ┃ ┗ 📜mydataset.yaml
 ┣ 📂models/
 ┃ ┣ 📜my-yolo26m-model.onnx
 ┃ ┗ 📜my-yolo26m-model.pt
 ┣ 📂src/
 ┃ ┗ 📜main.py
 ┣ 📜docker-compose.yml
 ┣ 📜Dockerfile
 ┗ 📜requirements.txt
      Dockerfile
      FROM ultralytics/ultralytics:latest-jetson-jetpack5

WORKDIR /app

# Copy the test app
COPY src/ ./src/

# Run the test
CMD ["python3", "src/main.py"]
      docker-compose.yml
      version: '3.8'

services:
  obb-test-jp5:
    image: obb-test:latest
    container_name: obb_test_jp5
    build:
      context: .
      dockerfile: Dockerfile
    runtime: nvidia
    network_mode: host
    ipc: host
    restart: "no"
    volumes:
      - ./models:/app/models
      - ./data:/app/data
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=all
      main.py
      """
Minimal OBB model test script.

Behavior:
- Runs `yolo checks` to output environment details.
- Uses files in `/app/models` (container path mapped from `obb-test/models`).
- Expects two files (hard-coded):
  - `/app/models/my-yolo26m-model.pt` (optional) — used for Ultralytics native export
  - `/app/models/my-yolo26m-model.onnx` (optional) — used for trtexec compilation
- Runs both export methods (if files exist) and validates inference on each compiled engine using mydataset dataset.
"""

import logging
import sys
import subprocess
import os
from pathlib import Path
from ultralytics import YOLO

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)


def run_yolo_checks():
    """Execute yolo checks via CLI to log environment details."""
    logger.info("=" * 60)
    logger.info("Environment Verification (yolo checks)")
    logger.info("=" * 60)
    try:
        res = subprocess.run(["yolo", "checks"], capture_output=True, text=True, check=True)
        logger.info("\n" + res.stdout.strip())
    except subprocess.CalledProcessError as e:
        logger.error(f"yolo checks failed:\n{e.stderr}")
    except FileNotFoundError:
        logger.error("yolo CLI command not found. Ensure ultralytics is installed in this environment.")


def test_export_to_engine():
    models_dir = Path("/app/models")
    pt_path = models_dir / "my-yolo26m-model.pt"
    onnx_path = models_dir / "my-yolo26m-model.onnx"
    dataset_yaml = "/app/data/mydataset.yaml"

    logger.info("=" * 60)
    logger.info("Minimal OBB Test - Export & Validation")
    logger.info("=" * 60)

    engines_tested = []

    # Part 1: Ultralytics native export (PT -> Engine)
    if pt_path.exists():
        try:
            logger.info(f"\nRunning Ultralytics native export for: {pt_path.name}")
            model = YOLO(str(pt_path))
            exported = model.export(format="engine", imgsz=1024, device=0, half=True)
            
            # Resolve the exported engine path
            native_engine = str(exported) if isinstance(exported, (str, Path)) else str(pt_path.with_suffix('.engine'))
            
            if os.path.exists(native_engine):
                logger.info(f"Ultralytics export successfully produced: {native_engine}")
                engines_tested.append(native_engine)
            else:
                logger.error(f"Ultralytics export completed, but engine file was not found at {native_engine}")
        except Exception as e:
            logger.error(f"Ultralytics native export failed: {e}", exc_info=True)
    else:
        logger.info(f"PT file not found ({pt_path}); skipping Ultralytics native export.")

    # Part 2: trtexec compilation (ONNX -> Engine)
    if onnx_path.exists():
        try:
            logger.info(f"\nRunning trtexec to compile ONNX: {onnx_path.name}")
            trt_engine = str(onnx_path.with_suffix('.trt.engine'))
            cmd = [
                "/usr/src/tensorrt/bin/trtexec",
                f"--onnx={onnx_path}",
                f"--saveEngine={trt_engine}",
                "--fp16",
                "--memPoolSize=workspace:768", # Prevents Out of Memory
            ]
            logger.info(f"Running command: {' '.join(cmd)}")
            res = subprocess.run(cmd, capture_output=True, text=True)
            
            if res.returncode != 0:
                logger.error("trtexec compilation failed.")
                logger.error(f"trtexec stderr:\n{res.stderr}")
            else:
                if os.path.exists(trt_engine):
                    logger.info(f"trtexec successfully saved engine to {trt_engine}")
                    engines_tested.append(trt_engine)
                else:
                    logger.error(f"trtexec returned success, but engine file was not found at {trt_engine}")
        except Exception as e:
            logger.error(f"trtexec execution failed: {e}", exc_info=True)
    else:
        logger.info(f"ONNX file not found ({onnx_path}); skipping trtexec compilation.")

    if not engines_tested:
        logger.error("\nNo engines are available to test. Exiting validation phase.")
        return False

    # Part 3: Inference Validation
    validation_success = True
    for eng in engines_tested:
        try:
            logger.info(f"\nValidating inference with engine: {eng}")
            
            model = YOLO(eng, task="obb")
            metrics = model.val(data=dataset_yaml, imgsz=1024, batch=1, verbose=False)
            
            logger.info(f"Validation metrics for {eng}: mAP50-95={metrics.box.map:.4f}, mAP50={metrics.box.map50:.4f}")
        except Exception as e:
            logger.error(f"Validation failed on {eng}: {e}", exc_info=True)
            validation_success = False

    logger.info("\nExecution complete.")
    return validation_success


if __name__ == "__main__":
    run_yolo_checks()
    
    try:
        success = test_export_to_engine()
        logger.info("\n" + "=" * 60)
        if success:
            logger.info("✓ Test passed")
        else:
            logger.info("✗ Test failed")
        logger.info("=" * 60)
        sys.exit(0 if success else 1)
    except Exception as e:
        logger.error(f"Unhandled script exception: {e}", exc_info=True)
        sys.exit(1)
      Output
      obb_test_jp5  | Creating new Ultralytics Settings v0.0.6 file ✅ 
obb_test_jp5  | View Ultralytics Settings with 'yolo settings' or at '/root/.config/Ultralytics/settings.json'
obb_test_jp5  | Update Settings with 'yolo settings key=value', i.e. 'yolo settings runs_dir=path/to/dir'. For help see https://docs.ultralytics.com/quickstart/#ultralytics-settings.
obb_test_jp5  | 2026-05-14 12:51:35,392 - INFO - ============================================================
obb_test_jp5  | 2026-05-14 12:51:35,392 - INFO - Environment Verification (yolo checks)
obb_test_jp5  | 2026-05-14 12:51:35,392 - INFO - ============================================================
obb_test_jp5  | 2026-05-14 12:51:50,554 - INFO - 
obb_test_jp5  | Ultralytics 8.4.50 🚀 Python-3.8.10 torch-2.1.0a0+41361538.nv23.06 CUDA:0 (Xavier, 6833MiB)
obb_test_jp5  | Setup complete ✅ (2 CPUs, 6.7 GB RAM, 91.7/107.6 GB disk)
obb_test_jp5  | 
obb_test_jp5  | OS                     Linux-5.10.216-tegra-aarch64-with-glibc2.29
obb_test_jp5  | Environment            Docker
obb_test_jp5  | Python                 3.8.10
obb_test_jp5  | Install                git
obb_test_jp5  | Path                   /ultralytics/ultralytics
obb_test_jp5  | RAM                    6.67 GB
obb_test_jp5  | Disk                   91.7/107.6 GB
obb_test_jp5  | CPU                    ARMv8 Processor rev 0 (v8l)
obb_test_jp5  | CPU count              2
obb_test_jp5  | GPU                    Xavier, 6833MiB
obb_test_jp5  | GPU count              1
obb_test_jp5  | CUDA                   11.4
obb_test_jp5  | 
obb_test_jp5  | numpy                  ✅ 1.23.5>=1.23.0
obb_test_jp5  | matplotlib             ✅ 3.7.5>=3.3.0
obb_test_jp5  | opencv-python-headless ✅ 4.13.0.92>=4.6.0
obb_test_jp5  | pillow                 ✅ 10.4.0>=7.1.2
obb_test_jp5  | pyyaml                 ✅ 6.0.3>=5.3.1
obb_test_jp5  | requests               ✅ 2.32.4>=2.23.0
obb_test_jp5  | scipy                  ✅ 1.10.1>=1.4.1
obb_test_jp5  | torch                  ✅ 2.1.0a0+41361538.nv23.6>=1.8.0
obb_test_jp5  | torchvision            ✅ 0.16.2+c6f3977>=0.9.0
obb_test_jp5  | psutil                 ✅ 7.2.2>=5.8.0
obb_test_jp5  | polars                 ✅ 1.8.2>=0.20.0
obb_test_jp5  | ultralytics-thop       ✅ 2.0.19>=2.0.18
obb_test_jp5  | torch                  ✅ 2.1.0a0+41361538.nv23.6!=2.4.0,>=1.8.0; sys_platform == "win32"
obb_test_jp5  | 2026-05-14 12:51:50,562 - INFO - ============================================================
obb_test_jp5  | 2026-05-14 12:51:50,563 - INFO - Minimal OBB Test - Export & Validation
obb_test_jp5  | 2026-05-14 12:51:50,564 - INFO - ============================================================
obb_test_jp5  | 2026-05-14 12:51:50,565 - INFO - 
obb_test_jp5  | Running Ultralytics native export for: my-yolo26m-model.pt
obb_test_jp5  | 2026-05-14 12:51:51,353 - ERROR - Ultralytics native export failed: ERROR ❌️ /app/models/my-yolo26m-model.pt requires numpy>=1.26.1, however numpy==1.23.5 is installed.
obb_test_jp5  | Traceback (most recent call last):
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/tasks.py", line 1466, in torch_safe_load
obb_test_jp5  |     ckpt = _load()
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/tasks.py", line 1463, in _load
obb_test_jp5  |     return torch_load(file, map_location="cpu")
obb_test_jp5  |   File "/ultralytics/ultralytics/utils/patches.py", line 181, in torch_load
obb_test_jp5  |     return torch.load(*args, **kwargs)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 877, in load
obb_test_jp5  |     return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 1253, in _load
obb_test_jp5  |     result = unpickler.load()
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 1246, in find_class
obb_test_jp5  |     return super().find_class(mod_name, name)
obb_test_jp5  | ModuleNotFoundError: No module named 'numpy._core'
obb_test_jp5  | 
obb_test_jp5  | The above exception was the direct cause of the following exception:
obb_test_jp5  | 
obb_test_jp5  | Traceback (most recent call last):
obb_test_jp5  |   File "src/minimal.py", line 58, in test_export_to_engine
obb_test_jp5  |     model = YOLO(str(pt_path))
obb_test_jp5  |   File "/ultralytics/ultralytics/models/yolo/model.py", line 76, in __init__
obb_test_jp5  |     super().__init__(model=model, task=task, verbose=verbose)
obb_test_jp5  |   File "/ultralytics/ultralytics/engine/model.py", line 144, in __init__
obb_test_jp5  |     self._load(model, task=task)
obb_test_jp5  |   File "/ultralytics/ultralytics/engine/model.py", line 283, in _load
obb_test_jp5  |     self.model, self.ckpt = load_checkpoint(weights)
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/tasks.py", line 1537, in load_checkpoint
obb_test_jp5  |     ckpt, weight = torch_safe_load(weight)  # load ckpt
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/tasks.py", line 1489, in torch_safe_load
obb_test_jp5  |     raise ModuleNotFoundError(
obb_test_jp5  | ModuleNotFoundError: ERROR ❌️ /app/models/my-yolo26m-model.pt requires numpy>=1.26.1, however numpy==1.23.5 is installed.
obb_test_jp5  | 2026-05-14 12:51:51,382 - INFO - 
obb_test_jp5  | Running trtexec to compile ONNX: my-yolo26m-model.onnx
obb_test_jp5  | 2026-05-14 12:51:51,382 - INFO - Running command: /usr/src/tensorrt/bin/trtexec --onnx=/app/models/my-yolo26m-model.onnx --saveEngine=/app/models/my-yolo26m-model.trt.engine --fp16 --memPoolSize=workspace:768
obb_test_jp5  | 2026-05-14 13:21:39,214 - INFO - trtexec successfully saved engine to /app/models/my-yolo26m-model.trt.engine
obb_test_jp5  | 2026-05-14 13:21:39,217 - INFO - 
obb_test_jp5  | Validating inference with engine: /app/models/my-yolo26m-model.trt.engine
obb_test_jp5  | Ultralytics 8.4.50 🚀 Python-3.8.10 torch-2.1.0a0+41361538.nv23.06 CUDA:0 (Xavier, 6833MiB)
obb_test_jp5  | Loading /app/models/my-yolo26m-model.trt.engine for TensorRT inference...
obb_test_jp5  | [05/14/2026-13:21:41] [TRT] [I] Loaded engine size: 43 MiB
obb_test_jp5  | [05/14/2026-13:21:45] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +260, GPU +306, now: CPU 648, GPU 5277 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +82, GPU +156, now: CPU 730, GPU 5433 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +41, now: CPU 0, GPU 41 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +28, now: CPU 687, GPU 5422 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +32, now: CPU 687, GPU 5454 (MiB)
obb_test_jp5  | [05/14/2026-13:21:46] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +92, now: CPU 0, GPU 133 (MiB)
obb_test_jp5  | Setting batch=1 input of shape (1, 3, 1024, 1024)
obb_test_jp5  | 2026-05-14 13:21:54,562 - INFO - generated new fontManager
obb_test_jp5  | val: Fast image access ✅ (ping: 1.1±0.9 ms, read: 479.2±146.1 MB/s, size: 2563.4 KB)
val: Scanning /app/data/labels/rec26.cache... 2940 images, 26 backgrounds, 0 corrupt: 100% ━━━━━━━━━━━━ 2940/2940 14.4Mit/s 0.0s
obb_test_jp5  | 2026-05-14 13:22:01,033 - ERROR - Validation failed on /app/models/my-yolo26m-model.trt.engine: CUDA error: no kernel image is available for execution on the device
obb_test_jp5  | CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
obb_test_jp5  | For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
obb_test_jp5  | Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
obb_test_jp5  | Traceback (most recent call last):
obb_test_jp5  |   File "src/minimal.py", line 114, in test_export_to_engine
obb_test_jp5  |     metrics = model.val(data=dataset_yaml, imgsz=1024, batch=1, verbose=False)
obb_test_jp5  |   File "/ultralytics/ultralytics/engine/model.py", line 615, in val
obb_test_jp5  |     validator(model=self.model)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
obb_test_jp5  |     return func(*args, **kwargs)
obb_test_jp5  |   File "/ultralytics/ultralytics/engine/validator.py", line 210, in __call__
obb_test_jp5  |     model.warmup(imgsz=(1 if pt else self.args.batch, self.data["channels"], imgsz, imgsz))  # warmup
obb_test_jp5  |   File "/ultralytics/ultralytics/nn/autobackend.py", line 313, in warmup
obb_test_jp5  |     non_max_suppression(warmup_boxes)  # warmup NMS
obb_test_jp5  |   File "/ultralytics/ultralytics/utils/nms.py", line 154, in non_max_suppression
obb_test_jp5  |     i = torchvision.ops.nms(boxes, scores, iou_thres)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torchvision/ops/boxes.py", line 41, in nms
obb_test_jp5  |     return torch.ops.torchvision.nms(boxes, scores, iou_threshold)
obb_test_jp5  |   File "/usr/local/lib/python3.8/dist-packages/torch/_ops.py", line 646, in __call__
obb_test_jp5  |     return self._op(*args, **kwargs or {})
obb_test_jp5  | RuntimeError: CUDA error: no kernel image is available for execution on the device
obb_test_jp5  | CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
obb_test_jp5  | For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
obb_test_jp5  | Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
obb_test_jp5  | 
obb_test_jp5  | 2026-05-14 13:22:01,521 - INFO - 
obb_test_jp5  | Execution complete.
obb_test_jp5  | 2026-05-14 13:22:01,524 - INFO - 
obb_test_jp5  | ============================================================
obb_test_jp5  | 2026-05-14 13:22:01,526 - INFO - ✗ Test failed
obb_test_jp5  | 2026-05-14 13:22:01,529 - INFO - ============================================================
obb_test_jp5 exited with code 1

      Export Device

      This is a bit of info on where my ONNX files come from and what my PC environment looks like where I compiled them.

      Yolo Checks on the ONNX Export Device
      Ultralytics 8.4.48 🚀 Python-3.13.9 torch-2.11.0+cu130 CPU (AMD Ryzen 5 5600G with Radeon Graphics)
Setup complete ✅ (12 CPUs, 15.0 GB RAM, 744.1/878.1 GB disk)

OS                     Linux-6.17.0-23-generic-x86_64-with-glibc2.39
Environment            Linux
Python                 3.13.9
Install                git
Path                   /home/myuser/projects/myproject/.venv/lib/python3.13/site-packages/ultralytics
RAM                    15.01 GB
Disk                   744.1/878.1 GB
CPU                    AMD Ryzen 5 5600G with Radeon Graphics
CPU count              12
GPU                    None
GPU count              None
CUDA                   None

numpy                  ✅ 2.4.4>=1.23.0
matplotlib             ✅ 3.10.9>=3.3.0
opencv-python          ✅ 4.13.0.92>=4.6.0
pillow                 ✅ 12.2.0>=7.1.2
pyyaml                 ✅ 6.0.3>=5.3.1
requests               ✅ 2.33.1>=2.23.0
scipy                  ✅ 1.17.1>=1.4.1
torch                  ✅ 2.11.0>=1.8.0
torch                  ✅ 2.11.0!=2.4.0,>=1.8.0; sys_platform == "win32"
torchvision            ✅ 0.26.0>=0.9.0
psutil                 ✅ 7.2.2>=5.8.0
polars                 ✅ 1.40.1>=0.20.0
ultralytics-thop       ✅ 2.0.19>=2.0.18
      Export Code
      from ultralytics import YOLO

# 1. Load the model weights
model_path = "my-yolo26m-model.pt"
model = YOLO(model_path)

# 2. Export the model
# Configured for OBB (Oriented Bounding Boxes) and legacy TensorRT compatibility
success = model.export(
    format="onnx", 
    imgsz=1024,      # Target resolution
    opset=13,       # ONNX operator set version, also tried 12
    simplify=True,   # I hoped this would help
    task="obb",
    half=False,      # I tried this with True and False
    end2end=False    # For legacy TensorRT support...?
)

print(f"Export successful: {success}")
      YOLO26m-obb summary (fused): 175 layers, 23,443,236 parameters, 0 gradients, 71.5 GFLOPs

PyTorch: starting from 'my-yolo26m-model.pt' with input shape (1, 3, 1024, 1024) BCHW and output shape(s) (1, 6, 21504) (90.8 MB)

ONNX: starting export with onnx 1.21.0 opset 13...
ONNX: slimming with onnxslim 0.1.93...
ONNX: export success ✅ 2.5s, saved as 'my-yolo26m-model.onnx' (81.3 MB)

Export complete (4.3s)

      Old Container

      Jut in case it is relevant, this is the yolo checks of my self-made container which is based on l4t-ml.

      root@xaviernx-desktop:/app# yolo checks
WARNING ⚠️ user config directory '/root/.config/Ultralytics' is not writable, using '/tmp/Ultralytics'. Set YOLO_CONFIG_DIR to override.
Creating new Ultralytics Settings v0.0.6 file ✅ 
View Ultralytics Settings with 'yolo settings' or at '/tmp/Ultralytics/settings.json'
Update Settings with 'yolo settings key=value', i.e. 'yolo settings runs_dir=path/to/dir'. For help see https://docs.ultralytics.com/quickstart/#ultralytics-settings.
Ultralytics 8.4.49 🚀 Python-3.8.10 torch-2.0.0+nv23.05 CUDA:0 (Xavier, 6833MiB)
Setup complete ✅ (2 CPUs, 6.7 GB RAM, 84.9/107.6 GB disk)

OS                     Linux-5.10.216-tegra-aarch64-with-glibc2.29
Environment            Docker
Python                 3.8.10
Install                pip
Path                   /usr/local/lib/python3.8/dist-packages/ultralytics
RAM                    6.67 GB
Disk                   84.9/107.6 GB
CPU                    ARMv8 Processor rev 0 (v8l)
CPU count              2
GPU                    Xavier, 6833MiB
GPU count              1
CUDA                   11.4

numpy                  ✅ 1.23.5>=1.23.0
matplotlib             ✅ 3.7.5>=3.3.0
opencv-python          ❌ (not installed)>=4.6.0
pillow                 ✅ 10.2.0>=7.1.2
pyyaml                 ✅ 5.3.1>=5.3.1
requests               ✅ 2.32.4>=2.23.0
scipy                  ✅ 1.10.1>=1.4.1
torch                  ✅ 2.0.0+nv23.5>=1.8.0
torch                  ✅ 2.0.0+nv23.5!=2.4.0,>=1.8.0; sys_platform == "win32"
torchvision            ✅ 0.15.1a0+42759b1>=0.9.0
psutil                 ✅ 5.9.6>=5.8.0
polars                 ❌ (not installed)>=0.20.0
ultralytics-thop       ❌ (not installed)>=2.0.18
    3. Murat Raimbekov

      Thanks for the detailed MRE. I reproduced the numpy failure end-to-end with torch.save/torch.load and have a verified fix. There are two independent issues here.

      1. numpy._core / "requires numpy>=1.26.1" — root cause and fix

      Your .pt was saved on your PC with numpy 2.x / Python 3.13, so the checkpoint pickle embeds the numpy-2 numpy._core module path. JetPack 5 on Xavier NX runs Python 3.8, whose maximum numpy is 1.24.x, and Xavier cannot move to JetPack 6 (Orin-only). I verified that even numpy 1.26.x does not fully read such a checkpoint, so "upgrade numpy" is not a real option on JP5.

      The reliable fix is a small compatibility shim that aliases numpy._core to the legacy numpy.core, injected before the model is loaded. I confirmed this loads a numpy-2 torch.save checkpoint correctly under numpy 1.23.5 with the model and all fields intact. Put this at the very top of your main.py, before from ultralytics import YOLO:

      import sys, numpy, numpy.core
sys.modules["numpy._core"] = numpy.core
for _s in ("multiarray", "numeric", "umath", "_multiarray_umath"):
    try:
        sys.modules[f"numpy._core.{_s}"] = __import__(f"numpy.core.{_s}", fromlist=[_s])
    except Exception:
        pass

      Alternatively, fix the checkpoint once on the PC (where numpy 2.x can read it) so the device never needs the shim — load it and re-save it after converting the auxiliary numpy fields, then copy the new .pt to the Jetson.

      2. Empty boxes / CUDA error: no kernel image is available for execution on the device

      These are separate from numpy:

      • The trtexec-built engine produces no detections because a raw trtexec engine carries none of the Ultralytics export metadata, so the OBB decoding/post-processing is never applied. Build the engine on the Jetson with Ultralytics directly from the .pt (after applying the shim above) so metadata is embedded: YOLO("model.pt").export(format="engine", imgsz=1024, half=True, device=0).
      • no kernel image is available is raised inside torchvision.ops.nms on the GPU: the PyTorch/torchvision in ultralytics/ultralytics:latest-jetson-jetpack5 is built for Orin (SM 8.7), while Xavier NX is SM 7.2. That is exactly why YOLO11 OBB works in your older l4t-ml-based container (torch 2.0.0+nv23.5) but the official image fails. Recommended path: stay on the container that already works on your Xavier, pip install ultralytics opencv-python-headless there, and run the Ultralytics export(format="engine") + val() entirely on-device.

      TL;DR: the numpy error is a pickle module-path mismatch from the numpy-2 export PC (fix with the shim above — verified), and the empty/CUDA errors are a raw-trtexec metadata loss plus a torch/torchvision build that doesn't target Xavier's SM 7.2.

      1
    4. RotatingGiraffeGitHub

      Hi there, thank you for the help! I just recently got around to trying out the shim and now everything works just fine, thank you for your help!

  9. yashpatel2010GitHub

    How to run deepstream for YOLO26 models in my jetson orin nano developer kit?

    Is there any documentation for deepstream on jetson? I don't want any installation errors of version missmatch.

    So give me any perfect guide link so that I will not miss anything while installation and run inference on live RTSP streams

    3. yashpatel2010GitHub

      Thanks for your reply. Just onemore doubt to clear. docker pull nvcr.io/nvidia/deepstream:7.1-triton-multiarch

      if I use above docker container for my jetpack6.1 so it there is no issue of version missmatch of deepstream right?

      is it works very well with Yolo26 deepstream settings which is showing in this doc? https://docs.ultralytics.com/guides/deepstream-nvidia-jetson/

      what is your recommendation to run Multiple RTSP streams with deepstream with docker containers or without docker containers manual installation?

    4. Glenn JocherUltralytics

      Yes, nvcr.io/nvidia/deepstream:7.1-triton-multiarch is the correct DeepStream 7.1 Jetson container for JetPack 6.1, but it still depends on the host having the JetPack BSP and NVIDIA container runtime installed, so it lowers mismatch risk rather than removing it completely; for YOLO26 + RTSP, the DeepStream multi-stream section is the right reference, and I’d suggest native setup first since that’s the install path used in the guide, then move to Docker once your pipeline is stable. (docs.ultralytics.com)

    5. yashpatel2010GitHub(已编辑)

      Thanks. For native setup Is it good to install deepstream 7.1 while installing jetpack6.1 only?

  10. githubwaltGitHub

    Installing torch 2.11.0 from the official website pytorch.org on an AGX Orin 64Gb (dev kit) with JP6.2.2 or installing torch 2.10.0 and torchvision 0.25.0 following this page, give complaints about not being compatible with the Compute Capability (CC) of my machine which has 8.7, whatever that means. Only installing the previous versions torch 2.8.0 and torchvision 0.23.0 do work.

    I understand that torch needs to be tailored specifically for the Jetson family and that is what this page is all about, so why do I have this issue if I follow the instructions given here? Is it because I am on Jetpack 6.2.2 and here it's 6.1?

    1. githubwaltGitHub

      I forgot the warning and error output when running a python test script:

      UserWarning: Found GPU0 Orin which is of compute capability (CC) 8.7.The following list shows the CCs this version of PyTorch was built for and the hardware CCs it supports:- 8.0 which supports hardware CC >=8.0,<9.0 except {8.7}- 9.0 which supports hardware CC >=9.0,<10.0

      torch.AcceleratorError CUDA error: no kernel image is available for execution on the device

    2. Glenn JocherUltralytics

      Thanks, that traceback helps — CC 8.7 is the AGX Orin GPU architecture, and no kernel image is available means the installed PyTorch wheel was built without sm_87 support. So yes, this is most likely a JetPack/PyTorch wheel mismatch: the native steps in the JetPack 6.1 section don’t necessarily map cleanly to JP6.2.2, even though the page includes JP6.2 benchmark coverage. For now I’d stick with the working torch 2.8.0 + torchvision 0.23.0 pair or use the JetPack 6 Docker setup; if you want us to check your exact environment, please post yolo checks and the full text traceback here in the discussion.

    4. Glenn JocherUltralytics

      No worries — the duplicate is fine, and if you later confirm a stable JP6.2.2 + PyTorch pairing on AGX Orin, please add it here so we can reflect it in the Jetson guide.

    5. githubwaltGitHub

      Will do. Now trying to get the most out of the AGX. Experimenting with number of workers on a batch size of 16. So far 12 workers gave memory issues ... I chose 12 because of 12 cores, but that was a little too much. Now running with 8 workers ... so far so good. To be continued 🙂

    6. githubwaltGitHub

      Hi Glenn, I can confirm a stable JP6.2.2 + Pytorch (torch 2.8.0 and torchvision 0.23.0) on the AGX Orin.