开始使用

Rockchip RKNN 导出

RKNN 是 Rockchip 处理器(如 RK3588 和 RK3566)中 NPU 的模型格式。使用 format="rknn" 导出 Ultralytics YOLO26 模型,会创建一个包含 .rknn 文件的 yolo26n_rknn_model/ 目录。该文件针对一款 Rockchip 芯片构建,你可以通过 name 参数选择芯片,并选择 FP16 或 INT8 精度。

概览#

属性详情
format 参数rknn
输出yolo26n_rknn_model/,一个包含 yolo26n-rk3588.rknn 和 metadata.yaml 的目录;检测、分割、姿态和 OBB 模型的 INT8 导出还会添加 tail.onnx
运行平台每次导出针对一款 Rockchip 芯片的 NPU:RK3588、RK3576、RK3566、RK3568、RK3562、RK2118、RV1126B、RV1103、RV1106、RV1103B 或 RV1106B,详见目标芯片。
运行时Rockchip 设备上的 RKNN Toolkit Lite2(rknn-toolkit-lite2 Python 包)
精确率FP16(默认)和 INT8;RV1103、RV1106、RV1103B 和 RV1106B 仅支持 INT8。不支持 FP32 导出。RK3588 上的 YOLO26n 模型大小为 FP16 时 7.4 MB,INT8 时 4.3 MB
导出主机Linux x86-64,搭配 Python 3.12 或更早版本。在 PC 上导出,然后将导出目录复制到 Rockchip 设备
在云端导出

Ultralytics Platform 可以将训练好的模型导出为针对你所选 Rockchip 芯片的 RKNN 格式,因此你无需准备本地导出环境。

快速入门#

在 x86-64 Linux 机器上创建独立虚拟环境并安装 Ultralytics 包,同时安装 RKNN 工具包所需的 protobuf 和 numpy 版本:

pip install ultralytics "protobuf<7" "numpy<2.4"

如果缺少相关依赖,首次导出会安装 onnx、onnxslim、onnxruntime 和 rknn-toolkit2 及其依赖项。使用普通的 pip install ultralytics 时,如果 numpy、protobuf 和 setuptools 的版本不在以下范围内,首次导出还会替换这些包,因此请使用独立环境。如果需要替换 protobuf 或 numpy,该次导出会失败,但在新进程中再次导出即可成功。导出需要满足以下版本要求:

软件包所需版本
Python3.12 或更早版本,这是 rknn-toolkit2 2.3.2 提供构建版本支持的最新版本
rknn-toolkit22.3.2,这是 PyPI 上唯一符合所需范围的版本(2.3.2 或更高版本,且低于 2.5)
onnx1.16.1 或更高版本,且低于 1.19.0
numpy低于 2.4
protobuf低于 7
setuptools低于 82

其他安装方式请参阅安装指南。然后在 PC 上导出模型,将导出目录复制到 Rockchip 设备,并在设备上加载模型以运行推理或验证精度。

导出
from ultralytics import YOLO

# 加载 YOLO26 模型
model = YOLO("yolo26n.pt")

# 将模型以 RKNN 格式导出为适用于 RK3588 的 FP16 模型
model.export(format="rknn", name="rk3588")  # creates 'yolo26n_rknn_model/'

# 也可以使用校准数据导出 INT8 量化的 RKNN 模型
model.export(format="rknn", name="rk3588", quantize=8, data="coco8.yaml")  # creates 'yolo26n_rknn_model/'

两种导出都会写入同一个 yolo26n_rknn_model/ 目录,因此第二次导出会替换第一次导出的内容。预测和验证只能在 Rockchip 设备上运行:

预测
from ultralytics import YOLO

# 在 Rockchip 设备上加载导出的 RKNN 模型
model = YOLO("yolo26n_rknn_model")

# 运行推理
results = model("https://ultralytics.com/images/bus.jpg")
验证
from ultralytics import YOLO

# 在 Rockchip 设备上加载导出的 RKNN 模型
model = YOLO("yolo26n_rknn_model")

# 在 COCO8 数据集上验证准确率
metrics = model.val(data="coco8.yaml")

支持的任务#

RKNN 导出支持 YOLOv8、YOLO11 和 YOLO26 模型的全部七种 Ultralytics 任务:目标检测、实例分割、语义分割、深度估计、分类、姿态估计 和 OBB。语义分割和深度估计仅适用于 YOLO26,因为只有 YOLO26 系列提供这两种任务头。

导出参数#

参数类型默认值说明
formatstr'rknn'导出模型的目标格式,用于定义与各种部署环境的兼容性。
namestr'rk3588'模型所针对的 Rockchip 芯片名称,大小写均可。请参阅目标芯片。
imgszint 或 tuple640模型输入图像尺寸:正方形图像可使用整数,也可以使用元组 (height, width)。未指定时,导出会使用检查点训练时的尺寸。官方 YOLO26 检查点中,检测、分割和姿态模型以及在 ADE20K 上预训练的语义分割模型使用 640;OBB 模型以及在 Cityscapes 上预训练的语义分割模型使用 1024;深度模型使用 768;分类模型使用 224。
quantizeint 或 strNone精度:16 或不设置时构建 FP16 模型;8 则使用 data 进行校准并构建 INT8 模型。对于仅支持 INT8 的芯片,不设置时会构建 INT8 模型,而 16 会报错。32 会报错,使用 quantize=8 训练的检查点无法导出为 RKNN,详见限制和故障排除。此参数取代已弃用的 half/int8 标志。
datastrNone用于 INT8 校准的数据集 YAML;分类任务则使用数据集目录或内置数据集名称。除非设置了 split,否则校准会使用 val 划分。如果使用 quantize=8 时未指定该参数,Ultralytics 会使用适用于模型任务的小型默认数据集,例如用于检测的 coco8.yaml。
fractionfloat、int 或 list1.0校准子集:可以指定比例、图像数量(大于 1 的整数),或 [train, val, test] 列表;导出会使用列表中的校准集条目。
batchint1导出模型的批次大小,该值固定在输入形状中。
simplifyboolTrue使用 onnxslim 简化中间 ONNX 图。
opsetintNone中间 ONNX 图的 ONNX opset 版本。未设置时使用 19,更高的值会被降至 19。
nmsbool,可选NoneRKNN 模型始终返回原始预测结果,由外部 NMS 处理。无 NMS 任务头(False)和嵌入式 NMS(True)均不可用,详见限制和故障排除。
devicestrNone用于追踪中间 ONNX 图的设备:cpu(默认值),或用于 CUDA GPU 的 0。

所有格式共有的参数请参阅导出模式文档。

目标芯片#

每个 .rknn 文件都针对一款芯片构建,并通过 name 选择芯片。以下尺寸对应于使用 quantize 未设置、以 imgsz=640 导出的 YOLO26n 检测模型:

name精度不设置 quantize 时构建YOLO26n 文件大小
rk3588FP16、INT8FP16yolo26n-rk3588.rknn7.4 MB
rk3576FP16、INT8FP16yolo26n-rk3576.rknn10.5 MB
rk3566FP16、INT8FP16yolo26n-rk3566.rknn6.8 MB
rk3568FP16、INT8FP16yolo26n-rk3568.rknn6.8 MB
rk3562FP16、INT8FP16yolo26n-rk3562.rknn6.8 MB
rk2118FP16、INT8FP16yolo26n-rk2118.rknn6.8 MB
rv1126bFP16、INT8FP16yolo26n-rv1126b.rknn7.1 MB
rv1103INT8INT8yolo26n-rv1103.rknn4.2 MB
rv1106INT8INT8yolo26n-rv1106.rknn4.2 MB
rv1103bINT8INT8yolo26n-rv1103b.rknn4.2 MB
rv1106bINT8INT8yolo26n-rv1106b.rknn4.2 MB

这四款仅支持 INT8 的芯片使用表中的默认 coco8.yaml 进行校准,其导出还会写入 tail.onnx。

量化#

quantize 用于选择 .rknn 文件的精度。以下尺寸对应于使用 imgsz=640 和 batch=1 导出的模型:

quantize精确率校准数据YOLO26n 和 rk3588 的文件大小
未设置或 16FP16否yolo26n-rk3588.rknn7.4 MB
8INT8是yolo26n-rk3588.rknn 和 tail.onnx4.3 MB 和 0.2 MB
  • FP16:支持 FP16 的七款芯片默认使用此精度。无需校准数据。
  • INT8:quantize=8 会将校准图像输入模型,并以 8 位整数存储权重和激活值。导出会使用 data 的 val 划分进行校准,或使用你通过 split 设置的划分;图像少于 300 张时会发出警告。使用 fraction 限制图像数量。
  • 混合精度:对于支持 FP16 的七款芯片,INT8 导出时,如果量化导致某层的输出变化过大,RKNN Toolkit 会保留该层的 FP16 精度(auto_hybrid)。四款仅支持 INT8 的芯片会跳过此步骤。
  • 检测头:对于检测、分割、姿态和 OBB 模型,INT8 .rknn 文件在检测头的输出卷积处结束。将这些输出解码为边界框的操作会保存到 tail.onnx 中,并以浮点数形式在设备 CPU 上运行,因为逐张量 INT8 无法表示边界框解码所需的像素范围数值。
  • 深度模型:深度模型的 INT8 导出会使用配有深度图的深度数据集图像进行校准。选择覆盖部署场景近距离和远距离范围的校准图像。使用默认的 depth8.yaml(含 4 张验证图像)时,rknn-toolkit2 模拟器中的 YOLO26n-depth INT8 模型对一张街景图像预测的深度最大为 3.9,而 FP16 模型的最大值为 13.9。
  • 设备端状态:所有任务的 INT8 导出都已在 rknn-toolkit2 模拟器中运行,并使用单张图像与 ONNX 模型进行比较。尚未在数据集或 Rockchip 硬件上测量其精度,因此部署前请在你的设备和自有数据上验证 INT8 模型。
示例
from ultralytics import YOLO

model = YOLO("yolo26n.pt")

# 使用 COCO 验证图像的 10%(500 张图像)进行校准;将 coco.yaml 替换为你自己的数据集
model.export(format="rknn", name="rk3588", quantize=8, data="coco.yaml", fraction=0.1)

输出格式#

导出会写入一个目录:

文件内容
yolo26n-rk3588.rknn模型文件名中包含由 name 设置的芯片名称
tail.onnx仅适用于 INT8 检测、分割、姿态和 OBB 模型:将检测头输出解码为原始预测结果的操作,运行于 CPU 上
metadata.yaml类别名称、任务、图像尺寸、批次大小、步幅和导出参数;Ultralytics 加载模型时会读取这些信息

中间 yolo26n.onnx 文件会保留在目录旁边。.rknn 模型接收一个形状为 (batch, height, width, 3) 的输入:RGB、通道后置(NHWC),uint8,取值范围为 0 到 255。模型内部会除以 255,因此无需自行缩放图像。这与 ONNX 导出不同,ONNX 导出接收通道前置的 float32 图像,且图像已缩放至 0-1。

输出取决于任务和精度。以下形状对应官方 YOLO26n 检查点在默认导出尺寸下的输出;语义分割使用 Cityscapes 检查点 yolo26n-sem.pt:

任务默认 imgszFP16 输出;INT8 输出为 tail.onnxINT8 .rknn 输出
检测640(1, 84, 8400)6 个任务头张量,由 tail.onnx 解码
分割640(1, 116, 8400) 和 (1, 32, 160, 160)9 个任务头张量和 (1, 32, 160, 160),由 tail.onnx 解码
语义1024(1, 19, 1024, 1024),每个类别和像素对应一个分数与 FP16 相同
深度768(1, 1, 768, 768),深度图与 FP16 相同
分类224(1, 1000),类别概率与 FP16 相同
姿态640(1, 56, 8400)9 个任务头张量,由 tail.onnx 解码
OBB1024(1, 20, 21504)9 个任务头张量,由 tail.onnx 解码
  • 原始预测结果:FP16 模型以及 INT8 模型的 tail.onnx 会返回与默认 ONNX 导出相同的原始预测结果:对于每个锚点,依次给出输入像素坐标中的边界框中心 x、中心 y、宽度和高度,然后是类别分数,最后是任务对应的数值(32 个掩码系数、17 × 3 个关键点数值,或以弧度表示的 OBB 角度)。你需要自行应用 NMS。各任务的输出布局请参阅 ONNX 输出格式。
  • INT8 任务头张量:INT8 检测模型会为三个检测尺度各返回两个张量,一个包含边界框数值,另一个包含类别分数;分割、姿态和 OBB 模型则会为每个尺度额外返回一个包含附加数值的张量。按照模型返回的顺序将这些张量传入 tail.onnx;分割模型还会通过 tail.onnx 原样传递掩码原型。
  • 语义分割:输出包含每个类别和像素对应的分数,而非 ONNX 导出的类别图,因此需要在你自己的代码中沿类别轴取 argmax。
  • NMS:不支持无 NMS 或嵌入式 NMS 输出,详见限制和故障排除。

YOLO("yolo26n_rknn_model") 会从 metadata.yaml 读取布局,运行 tail.onnx 并应用 NMS,因此只有在你自己的代码中运行模型时才需要了解这些细节。metadata.yaml 的键包括 names、task、imgsz、stride、batch、end2end、args,以及姿态模型的 kpt_shape。

使用 RKNN Toolkit Lite2 运行#

如果要在设备上脱离 Ultralytics 包运行模型,请使用 rknn-toolkit-lite2 包加载模型;对于 INT8 检测、分割、姿态或 OBB 模型,请通过 ONNX Runtime 将输出传入 tail.onnx:

# Export first: yolo export model=yolo26n.pt format=rknn name=rk3588 quantize=8 data=coco8.yaml
import numpy as np
import onnxruntime as ort
from rknnlite.api import RKNNLite

rknn = RKNNLite()
assert rknn.load_rknn("yolo26n_rknn_model/yolo26n-rk3588.rknn") == 0
assert rknn.init_runtime() == 0

image = np.zeros((1, 640, 640, 3), dtype=np.uint8)  # 你的 letterbox 处理后图像:RGB、NHWC、0-255
outputs = rknn.inference(inputs=[image])  # INT8 检测模型的六个任务头张量

tail = ort.InferenceSession("yolo26n_rknn_model/tail.onnx", providers=["CPUExecutionProvider"])
feed = {t.name: x.astype(np.float32) for t, x in zip(tail.get_inputs(), outputs)}
predictions = tail.run(None, feed)[0]  # 形状 (1, 84, 8400),需要应用 NMS 的原始预测结果
assert predictions.shape == (1, 84, 8400), "export a detection model at imgsz=640 with quantize=8"

无论导出时使用何种 batch,tail.onnx 的批次大小都为 1,因此使用较大批次导出的模型时,每张图像都要单独运行一次。FP16 模型没有 tail.onnx,而 outputs[0] 已包含原始预测结果。

部署#

在 Rockchip 设备上,通过 pip install ultralytics 安装该包,并将 yolo26n_rknn_model/ 目录复制到设备上;Quickstart 中的 protobuf 和 numpy 版本仅在导出时需要。随后,YOLO("yolo26n_rknn_model") 会通过 rknn-toolkit-lite2 Python 包在 NPU 上运行模型;如果目录中包含 tail.onnx,Ultralytics 会在首次使用时安装该 Python 包以及 onnxruntime。设备需要使用 Python 3.12 或更早版本,这是 rknn-toolkit-lite2 提供构建版本支持的最新版本。

  • 需要 Rockchip 设备:Ultralytics 仅在 64 位 ARM Linux 系统上加载 RKNN 模型,且系统设备树中的 compatible 字符串必须以受支持的芯片名称之一结尾。其他环境下加载会失败,详见限制和故障排除。rknn-toolkit-lite2 也仅提供 64 位 ARM Linux 构建版本,因此在不满足这些条件的目标设备上,请使用 Rockchip 的 C API 运行 .rknn 文件,并运行 tail.onnx,或在你自己的代码中实现相同的解码逻辑。
  • 芯片匹配:导出时使用与你的开发板所用芯片对应的 name。
  • 输入形状固定:模型始终按照导出时的 imgsz 和 batch 运行,Ultralytics 会从 metadata.yaml 读取这些值。传入 predict 或 val 的 imgsz 会被忽略。较大的批次会拆分为多个导出批次大小的块,较小的批次则会填充,因此任意数量的图像都可以处理。
  • device:无需选择设备。.rknn 模型通过 RKNN 运行时运行,tail.onnx 则通过 CPU 上的 ONNX Runtime 运行。

此工作流的早期版本曾在 Radxa Rock 5B(RK3588)和 Radxa Zero 3W(RK3566)上进行测试。目前的 INT8 导出尚未在硬件上运行。

部署指南#

限制和故障排查#

  • 导出主机:在 x86-64 Linux 机器上导出。在 macOS 上,导出无法安装 rknn-toolkit2,并会因 No module named 'rknn' 失败;该包没有 macOS 或 Windows 构建版本。在 ARM64 Linux(包括 Rockchip 开发板)上导出尚未经过测试,也不受支持:Ultralytics 仅在 x86-64 Linux 上测试 RKNN 导出。

  • Python 版本:rknn-toolkit2 2.3.2 支持 Python 3.12 及更早版本。在较新版本的 Python 上,请使用 Python 3.12 创建环境。

  • 在新环境中首次导出:如果环境中没有 Quickstart 指定的 protobuf 和 numpy 版本,首次导出会在运行期间替换这些包并失败。如果替换的是 protobuf 7(pip 会将其与 onnx 一同安装),错误为 AttributeError: 'google._upb._message.FieldDescriptor' object has no attribute 'label';如果只替换 numpy,错误为 TypeError: only 0-dimensional arrays can be converted to Python scalars。在新进程中再次运行导出(在 notebook 中即重启内核),或者像 Quickstart 中那样在首次导出前安装 "protobuf<7" "numpy<2.4"。

  • RKNN 模型无法在 PC 上运行:在 PC 上使用 YOLO("yolo26n_rknn_model") 进行预测或验证会因 RKNN inference is only supported on Rockchip devices. 失败;如果开发板的 /proc/device-tree/compatible 字符串不以受支持的芯片名称之一结尾,也会失败。使用 cat /proc/device-tree/compatible 输出该字符串。

  • FP32:quantize=32 会因 quantize=32 (FP32) is not supported; format='rknn' supports quantize=16 (FP16), 8 (INT8). 失败

  • 仅支持 INT8 的芯片:对于 rv1103、rv1106、rv1103b 和 rv1106b,未设置 quantize 的导出会记录 Rockchip target 'rv1106' requires INT8 quantization, enabling it. 并构建 INT8 模型;quantize=16 则会因 Rockchip target 'rv1106' only supports INT8, but got quantize=16. 失败

  • 缺少或错误的 name:未提供 name 时,导出会记录 Rockchip RKNN export requires a missing 'name' arg for processor type. Using default name='rk3588'.。值无效时,会先因 Invalid processor name 'rk9999' for Rockchip RKNN export. 失败,然后列出有效名称。

  • 无 NMS 任务头:RKNN 模型始终返回需要应用 NMS 的原始预测结果。使用 nms=False 时,导出 YOLO26 或 YOLOv10 模型会记录 This export format does not support end2end models, disabling the end2end branch. 并使用一对多任务头;导出其他检测、分割、姿态和 OBB 模型(例如 YOLO11)时,会记录 This model has no one-to-one head; using one-to-many outputs.。分类、语义分割和深度模型会忽略 nms=False。

  • 嵌入式 NMS:nms=True 会记录 format=rknn does not support embedded NMS; exporting native outputs for external NMS. 并写入原始输出。

  • 使用 quantize=8 训练的检查点:经过量化感知训练微调的检查点会因 format='rknn' cannot export a QAT model: its Q/DQ ranges are only read by the 'onnx' and 'engine' backends 失败。请改用 quantize=8 导出开始微调前使用的检查点。

  • RT-DETR:使用 rknn-toolkit2 2.3.2 时,RT-DETR 模型的导出会在工具包内部失败,FP16 和 INT8 均如此;对于 rtdetr-l.pt,失败发生在 imgsz=640,错误为 ValueError: operands could not be broadcast together with shapes (256,256) (8400,1)。

  • 输入形状固定:dynamic=True 会因 argument 'dynamic' is not supported for format='rknn' 失败。重新导出以更改图像尺寸或批次大小。

  • 未指定 data 的校准:检测模型使用 quantize=8 导出时,会记录 INT8 export requires a missing 'data' arg for calibration. Using default 'data=coco8.yaml'. 并使用该小型数据集进行校准;其他任务会指定各自的默认数据集。请传入 data,并提供能代表部署数据的图像。

  • 校准数据量:校准集少于 300 张图像时,导出会记录 >300 images recommended for INT8 calibration, found 4 images.。coco8.yaml 只有 4 张验证图像,足以测试导出,但不足以为部署校准模型。

  • 输出目录:同一检查点的每次导出都会写入相同的 yolo26n_rknn_model/ 目录,无论其 name、quantize、imgsz 或 batch 如何设置,并会先删除目录中的原有内容。为第二款芯片或另一种精度导出前,请重命名目录,并在名称中保留 _rknn_model,例如 yolo26n_rk3566_rknn_model/,因为 Ultralytics 会通过它识别格式。

  • ONNX opset:高于 19 的 opset 会记录 rknn-toolkit2 requires opset<=19, setting opset=19.

  • 早期版本导出的 INT8 模型:请重新导出在 ultralytics 8.4.191 之前导出的 INT8 检测、分割、姿态和 OBB 模型;该版本新增了 tail.onnx。这些模型返回的框和关键点坐标是相对于输入尺寸的,而当前版本不再对其进行缩放,因此模型可以正常加载,但返回的框坐标是错误的。

  • 运行时版本不匹配:如果设备日志报告 RKNN 运行时版本与 RKNN Toolkit 版本不匹配,且推理失败,请将 /usr/lib/librknnrt.so 替换为官方的 librknnrt.so 文件。

    RKNN 日志警告模型工具包版本与 librknnrt 运行时版本不匹配

其他错误请参阅常见问题指南。

基准测试#

以下 YOLO26 基准测试由 Ultralytics 团队在基于 Rockchip RK3588 的 Radxa Rock 5B 上运行,采用 rknn 模型格式测量速度和准确率。

性能
模型格式精确率状态大小 (MB)mAP50-95(B)推理时间 (ms/im)
YOLO26nrknnFP16✅7.20.47758.5
YOLO26nrknnINT8✅4.10.46341.2
YOLO26srknnFP16✅21.00.56993.8
YOLO26srknnINT8✅12.00.55461.9
YOLO26mrknnFP16✅43.00.608226.3
YOLO26mrknnINT8✅22.00.603122.0
YOLO26lrknnFP16✅53.00.628264.7
YOLO26lrknnINT8✅28.00.617145.3
YOLO26xrknnFP16✅113.00.664655.4
YOLO26xrknnINT8✅58.00.649287.4

使用 ultralytics 8.4.60 进行基准测试

注意

上述基准测试使用 COCO128 数据集进行验证。推理时间不包括预处理和后处理。当前 INT8 导出路径(主机端解码和 auto_hybrid)在 Rockchip 硬件上完成验证后,将更新 INT8 相关行。

表格中的尺寸是在早期版本下记录的;本页其他位置的尺寸是当前导出文件 .rknn 的大小,单位为 MB(10^6 字节)。YOLO26 模型页面列出了 PyTorch 检查点在完整 COCO 验证集上的准确率。

关于 RKNN#

Rockchip 设计了内置 NPU 的 ARM 片上系统,应用于单板计算机以及 Radxa、ASUS、Pine64、Orange Pi、Odroid、Khadas 和 Banana Pi 等公司的嵌入式产品。RKNN 是 Rockchip 神经网络的简称,也是这些 NPU 使用的模型格式;RKNN Toolkit 可在 PC 上将模型转换为该格式,并在设备上运行模型。

Rockchip RKNN export for NPU deployment

常见问题#

  • 请使用开发板上处理器的名称,而不是开发板的名称:例如,Radxa Rock 5B 使用 rk3588,Radxa Zero 3W 使用 rk3566。在开发板上,cat /proc/device-tree/compatible 会输出 Ultralytics 用来读取芯片型号的字符串。针对一种芯片导出的模型无法用于另一种芯片,因此每种芯片都需要单独导出。可接受的名称请参阅目标芯片。

  • 建议从默认的 FP16 开始。它无需校准数据,也会直接返回原始预测结果,无需在 CPU 上运行第二个模型。如果你需要更小、更快的模型,或者芯片仅支持 INT8,则选择 INT8(quantize=8):RK3588 上的 YOLO26n 使用 INT8 时为 4.3 MB,而 FP16 时为 7.4 MB;在基准测试中(使用早期导出版本测量),INT8 运行 YOLO26n 每张图像耗时 41.2 ms,而 FP16 为 58.5 ms。INT8 需要有代表性的校准数据;对于检测、分割、姿态和 OBB 模型,它还会使用 CPU 和 tail.onnx 执行框解码。选择前,请在你的设备上测量延迟和准确率。

  • 不能使用 YOLO(),因为它只能在 Rockchip 设备上加载 RKNN 模型。若要在 PC 上检查模型本身,可以使用 ONNX Runtime 验证 PyTorch 检查点,或验证导出时生成在目录旁边的中间文件 yolo26n.onnx:RKNN 模型就是从该计算图转换而来的。这不能替代在设备上的验证,尤其是对于 INT8,因为量化会改变输出。

  • 可以,但需要使用各加速器对应的导出格式:RKNN 模型只能在 Rockchip NPU 上运行。NVIDIA Jetson 请参阅 TensorRT,此外还可参阅 Hailo、Axelera、DeepX 和 Coral Edge TPU;概览请参阅模型部署选项指南。

评论45

提出问题、分享结果,或帮助其他人。

  1. as-jacksonGitHub

    We are pleased to confirm that the following ArmSoM boards based on Rockchip RK3576 have been successfully tested with Ultralytics projects (e.g., YOLO11n):

    ArmSoM-Sige5 ArmSoM-CM5

    Validation Scope:

    Model training/inference pipeline

    1
  2. yizi72GitHub

    I tried to convert the yolo11n.pt into a rknn model through codes below: from ultralytics import YOLO model = YOLO("yolo11n.pt") model.export(format="rknn", name="rk3588") and everything went correct. A rknn model named yolo11n-rk3588.rknn was build and used to infer a very traditional yolo demo image "bus"(a blue bus with several people in front of it), but no detection result was obtained.

    I tried to convert the yolol11n.pt into onnx model through above codes firstly, and then used a convert.py offered by rknn_model_zoo/examples/yolo11/python,

    import sys from rknn.api import RKNN

    DATASET_PATH = '../../../datasets/COCO/coco_subset_20.txt' DEFAULT_RKNN_PATH = '/home/yizi/model/yolo11/yolodet.rknn' DEFAULT_QUANT = False

    def parse_arg(): if len(sys.argv) < 3: print("Usage: python3 {} onnx_model_path [platform] [dtype(optional)] [output_rknn_path(optional)]".format(sys.argv[0])) print(" platform choose from [rk3562,rk3566,rk3568,rk3588,rk3576,rk1808,rv1109,rv1126]") print(" dtype choose from [i8, fp] for [rk3562,rk3566,rk3568,rk3588,rk3576]") print(" dtype choose from [u8, fp] for [rk1808,rv1109,rv1126]") exit(1)

    model_path = sys.argv[1]
    platform = sys.argv[2]
    
    do_quant = DEFAULT_QUANT
    if len(sys.argv) > 3:
        model_type = sys.argv[3]
        if model_type not in ['i8', 'u8', 'fp']:
            print("ERROR: Invalid model type: {}".format(model_type))
            exit(1)
        elif model_type in ['i8', 'u8']:
            do_quant = True
        else:
            do_quant = False
    
    if len(sys.argv) > 4:
        output_path = sys.argv[4]
    else:
        output_path = DEFAULT_RKNN_PATH
    
    return model_path, platform, do_quant, output_path

    if name == 'main': model_path, platform, do_quant, output_path = parse_arg()

    # Create RKNN object
    rknn = RKNN(verbose=False)
    
    # Pre-process config
    print('--> Config model')
    rknn.config(mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform=platform)
    print('done')
    
    # Load model
    print('--> Loading model')
    ret = rknn.load_onnx(model=model_path)
    if ret != 0:
        print('Load model failed!')
        exit(ret)
    print('done')
    
    # Build model
    print('--> Building model')
    ret = rknn.build(do_quantization=do_quant, dataset=DATASET_PATH)
    if ret != 0:
        print('Build model failed!')
        exit(ret)
    print('done')
    
    # Export rknn model
    print('--> Export rknn model')
    ret = rknn.export_rknn(output_path)
    if ret != 0:
        print('Export rknn model failed!')
        exit(ret)
    print('done')
    
    # Release
    rknn.release()

    Still everything went correct. A rknn model was created and applied to "bus.jpg", and there was no detection at all. I wonder what's wrong in the whole process?

    1
    1. Glenn JocherUltralytics

      @yizi72 thanks for the detailed report. For RKNN inference issues with YOLO11, please ensure:

      1. You're using matching RKNN Toolkit and runtime versions (update both if needed)
      2. Input images are 640x640 with pixel values normalized 0-255
      3. You load models via YOLO('./yolo11n_rknn_model') as shown in our RKNN deployment docs

      If issues persist, please share your full environment details and validation results from both export methods.

    2. yizi72GitHub

      1、I used rknn-toolkit2-2.3.0 which is the latest version ,bu I still did "pip install rknn-toolkit2 --upgrade" again. 2、The input image is 640*640, uchar, 0-255. 3. I used the following setting to transfer yolo11n.pt to onnx, from ultralytics import YOLO

      加载一个模型,路径为 YOLO 模型的 .pt 文件

      model = YOLO("yolo11n.pt")

      导出模型,设置多种参数

      model.export( format="onnx", imgsz=(640, 640),
      keras=False,
      optimize=True,
      half=False,
      int8=False,
      dynamic=False,
      simplify=True,
      opset=12,
      #workspace=4.0,
      nms=False,
      batch=1
      ) and applied the following codes to verify yolo11n.onnx model:

      from ultralytics import YOLO from PIL import Image

      Load a model

      model = YOLO("/home/yizi/models/yolo11/yolo11n.onnx") # load an official model

      Predict with the model

      results = model("bus.jpg") # predict on an image #results = rknn_model("bus.jpg")

      Access the results

      for result in results: xywh = result.boxes.xywh # center-x, center-y, width, height xywhn = result.boxes.xywhn # normalized xyxy = result.boxes.xyxy # top-left-x, top-left-y, bottom-right-x, bottom-right-y xyxyn = result.boxes.xyxyn # normalized names = [result.names[cls.item()] for cls in result.boxes.cls.int()] # class name of each box confs = result.boxes.conf # confidence score of each box

      The results are good. All objects in bus.jpg were detected. Then I run the following code to transfer onnx to rknn:

      from rknn.api import RKNN import cv2 import numpy as np

      if name == 'main': rknn = RKNN(verbose=True,verbose_file="log.txt") rknn.config( mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform="rk3588", optimization_level=3 )

      rknn.load_onnx(
          model="/home/yizi/models/yolo11/yolo11n.onnx",
          input_size_list=[[1,3,640,640]], 

      )

      rknn.build(
          do_quantization=False 
      )
      
      rknn.export_rknn(
          export_path="/home/yizi/models/yolo11/yolo11n.rknn" 
      )
      
      rknn.release()

      The transferring process went smooth, two problems appeared.

      Firstly,when this rknn model was pushed into a rk3588 tablet, and run with a binary executable file generated by cross-compilation, no result was detected.

      Secondly, I tried to run the following codes:

      from ultralytics import YOLO from PIL import Image

      rknn_model = YOLO("/home/yizi/models/yolo11/yolo11n.rknn", task=detect)

      results = model("bus.jpg")

      An error message, "model='/home/yizi/models/yolo11/yolo11n.rknn' is not a supported model format. Ultralytics supports: ('PyTorch', 'TorchScript', 'ONNX', 'OpenVINO', 'TensorRT', 'CoreML', 'TensorFlow SavedModel', 'TensorFlow GraphDef', 'TensorFlow Lite', 'TensorFlow Edge TPU', 'TensorFlow.js', 'PaddlePaddle', 'MNN', 'NCNN', 'IMX', 'RKNN') See https://docs.ultralytics.com/modes/predict for help. " appeared.

      Last question, how many outputs does yolo11 yield? one or six?

    3. Glenn JocherUltralytics

      @yizi72 thank you for the detailed report. For RKNN deployment issues:

      1. When using Ultralytics RKNN export, load models via the exported directory (not .rknn file), as shown in our RKNN deployment docs:
      model = YOLO('./yolo11n_rknn_model')  # directory path
      1. Ensure your RK3588 runtime uses matching librknnrt.so versions with your RKNN Toolkit 2.3.0 installation

      2. YOLO11 detection models output 3 tensors by default (different from older YOLO versions). For conversion issues, we recommend using our native model.export(format='rknn') rather than manual ONNX conversion.

      Let me know if you still face issues after verifying these points.

    4. yizi72GitHub

      Thanks for replies. I did verify all points you've mentioned above. When running model("bus.jpg"), error messages show:

      File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/ultralytics/engine/model.py", line 182, in call return self.predict(source, stream, **kwargs) File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/ultralytics/engine/model.py", line 543, in predict self.predictor.setup_model(model=self.model, verbose=is_cli) File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/ultralytics/engine/predictor.py", line 378, in setup_model self.model = AutoBackend( File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/ultralytics/nn/autobackend.py", line 495, in init raise OSError("RKNN inference is only supported on Rockchip devices.") OSError: RKNN inference is only supported on Rockchip devices.

      It's very strange because I did connect a RK3588 tablet with my virtual machine.

      I still have a quesiton on yolo11n output: yolo11n.pt has one output with shape (1, 84, 8400), and yolo11n-rk3588.rknn has one output with shape (1,84, 8400, 0). Are they correct? If so, can I use the similar codes to analyze rknn output, such as :

      for result in results: xywh = result.boxes.xywh # center-x, center-y, width, height xywhn = result.boxes.xywhn # normalized xyxy = result.boxes.xyxy # top-left-x, top-left-y, bottom-right-x, bottom-right-y xyxyn = result.boxes.xyxyn # normalized names = [result.names[cls.item()] for cls in result.boxes.cls.int()] # class name of each box confs = result.boxes.conf

    5. Glenn JocherUltralytics

      @yizi72 thanks for the details. For the RKNN inference error: ensure your Python script is running directly on the RK3588 hardware (not through a VM) as RKNN models require native Rockchip NPU access. Regarding outputs, YOLO11 detection models produce 3 output tensors by default (see our RKNN deployment docs), but your code should work if using the exported RKNN directory with the included postprocessing. Let me know if you still face issues after verifying native hardware execution.

  3. Yufeng Wu

    Hello, Ultralytics team. I wonder if it's possible to port yolo26n-pose to rknn? I'm not sure if it's difficult to work out on my own? Is there any model architecture problems that prevent porting the pose models to rknn?

  4. montemadjoGitHub

    It appears it does not support rv1106 platform yet

    When I try to export the model using this command

    model.export(format="rknn", name="rv1106", imgsz=320)

    It gives the following error

    E build: Current target_platform(rv1106) not support do_quantization = False!
    E export_rknn: RKNN model does not exist, please load & build model first!

    🤔

    1. Glenn JocherUltralytics(已编辑)

      rv1106 is an INT8-only target, so it needs a quantized RKNN build. Older Ultralytics releases always built with do_quantization=False, which fails with the message you saw. Current releases support it: upgrade with pip install -U ultralytics and export with quantize=8 plus calibration data:

      model.export(format="rknn", name="rv1106", imgsz=320, quantize=8, data="coco8.yaml")

      See the Rockchip RKNN export guide. If it still fails, share your !yolo checks output and rknn-toolkit2 version here and we can sanity-check the environment.

      1
    2. montemadjoGitHub(已编辑)

      Here's the quick workaround for those who want to work with yolo26 on RV1106 chip (like Luckfox pico etc). I ran into a couple of issues there, including quantization erasing my confidence results.

      Long story short - first you need to monkey-patch head.Detect.forward function in order to get two tensors - one for bboxes and one for scores. That way you escape all the heavy non-supported tensor functions. Then you export your yolo26 model to onnx format and finally to rknn like this

      import torch
      from ultralytics.nn.modules import head
      
      # 1. First we monkey-patch the forward function
      def patched_forward(self, x):
          if self.end2end:
              h = self.one2one
          else:
              h = self.one2many
      
          box_head = h["box_head"]
          cls_head = h["cls_head"]
      
          if box_head is None or cls_head is None:
              raise RuntimeError("Head moduls are None")
      
          bs = x[0].shape[0]
          boxes = torch.cat([box_head[i](x[i]).view(bs, 4 * self.reg_max, -1) for i in range(self.nl)], dim=-1)
          scores = torch.cat([cls_head[i](x[i]).view(bs, self.nc, -1) for i in range(self.nl)], dim=-1)
      
          preds = {"boxes": boxes, "scores": scores, "feats": x}
          y = self._inference(preds)
      
          # We are splitting into 2 tensors - both gets its own scale/zp at INT8 quantization
          bboxes = y[:, :4, :]   # [1, 4, 2100] — range 0-640
          scores = y[:, 4:, :]   # [1, 1, 2100] — range 0-1 (already sigmoid-ed)
          score_max_idx = scores[0, 0, :].argmax()
          return bboxes, scores
      
      head.Detect.forward = patched_forward
      
      # 2. Then we export to onnx model (we got best.pt model from ultralytics platform)
      from ultralytics import YOLO
      model = YOLO("best.pt")
      model.export(format="onnx", dynamic=False, opset=12, imgsz=320)
      
      # 3. Then we export to RKNN format
      from rknn.api import RKNN
      
      rknn = RKNN()
      
      # 3.1. Konfiguracija specifična za RV1106
      rknn.config(
          mean_values=[[0, 0, 0]], 
          std_values=[[255, 255, 255]], 
          target_platform='rv1106',
          quantized_algorithm='normal', # Ili 'mmse' za bolju preciznost
          quantized_dtype='asymmetric_quantized-8'
      )
      
      # 3.2. Load the ONNX model
      print("Loading ONNX model...")
      rknn.load_onnx(model='best.onnx')
      
      # 3.3. Build modela sa INT8 kvantizacijom
      # calib.txt should contain paths to calibration images
      print("Building RKNN model with quantization...")
      rknn.build(do_quantization=True, dataset='./calib.txt')
      
      # 3.4. Exporting final .rknn file
      print("Exporting RKNN model...")
      rknn.export_rknn('./yolo26_rv1106_model_optimized.rknn')
      
      rknn.release()

      After this you need to postprocess those two outputs like this

      yolo26_postprocess.h

      /**
       * YOLO26 Post-processing header for Fish Counter
       *
       * Supports YOLO26 2-output format: boxes [1, 4, N] + scores [1, nc, N]
       *
       * Based on the working rknn_minimal_test example.
       */
      
      #ifndef _YOLO26_POSTPROCESS_H_
      #define _YOLO26_POSTPROCESS_H_
      
      #include <stdint.h>
      #include <vector>
      #include "rknn_api.h"
      #include "rknn_detector.h"
      
      // YOLO26 model configuration defaults
      #define YOLO26_CONF_THRESHOLD 0.25f
      #define YOLO26_NMS_THRESHOLD 0.45f
      
      // Detection result (same structure as postprocess.h for compatibility)
      typedef struct
      {
          float x1, y1, x2, y2; // Bounding box (in model coordinates)
          float confidence;
          int class_id;
      } yolo26_detection_t;
      
      // YOLO26 context for format detection and caching
      typedef struct
      {
          int num_predictions;
          int num_classes;
          bool boxes_transposed; // true = [1, 4, N], false = [1, N, 4]
      } yolo26_context_t;
      
      /**
       * Initialize YOLO26 post-processing
       * Detects the output format from model attributes
       *
       * @param app_ctx The RKNN app context with model info
       * @param yolo26_ctx Output: YOLO26 context for format info
       * @return 0 on success, -1 on error
       */
      int yolo26_init_postprocess(rknn_app_context_t *app_ctx, yolo26_context_t *yolo26_ctx);
      
      /**
       * Deinitialize YOLO26 post-processing
       */
      void yolo26_deinit_postprocess();
      
      /**
       * YOLO26 post_process: processes model outputs and returns detections
       * Uses 2-output format: boxes [1, 4, N] + scores [1, nc, N]
       *
       * @param app_ctx The RKNN app context
       * @param yolo26_ctx YOLO26 context from init
       * @param outputs Pointer to output tensor memory array
       * @param letter_box Letterbox transform info for coordinate mapping
       * @param conf_threshold Confidence threshold
       * @param nms_threshold NMS threshold
       * @param od_results Output: detection results (compatible with existing code)
       * @return 0 on success, -1 on error
       */
      int yolo26_post_process(rknn_app_context_t *app_ctx, yolo26_context_t *yolo26_ctx,
                              void *outputs, letterbox_t *letter_box,
                              float conf_threshold, float nms_threshold,
                              object_detect_result_list *od_results);
      
      #endif // _YOLO26_POSTPROCESS_H_

      yolo26_postprocess.cc

      /**
       * YOLO26 Post-processing for Fish Counter on RV1106
       *
       * Supports YOLO26 2-output format: boxes [1, 4, N] + scores [1, nc, N]
       *
       * Based on the working rknn_minimal_test example.
       */
      
      #include "yolo26_postprocess.h"
      
      #include <math.h>
      #include <stdint.h>
      #include <stdio.h>
      #include <stdlib.h>
      #include <string.h>
      #include <algorithm>
      #include <vector>
      
      // Clamp value to range
      inline static int clamp(float val, int min, int max)
      {
          return val > min ? (val < max ? val : max) : min;
      }
      
      // Dequantize INT8 to float
      static inline float dequantize(int8_t qnt, int32_t zp, float scale)
      {
          return ((float)qnt - (float)zp) * scale;
      }
      
      // Sigmoid function
      static inline float sigmoid(float x)
      {
          return 1.0f / (1.0f + expf(-x));
      }
      
      // Calculate IoU between two boxes
      static float calculate_iou(float x1_a, float y1_a, float x2_a, float y2_a,
                                 float x1_b, float y1_b, float x2_b, float y2_b)
      {
          float x1 = std::max(x1_a, x1_b);
          float y1 = std::max(y1_a, y1_b);
          float x2 = std::min(x2_a, x2_b);
          float y2 = std::min(y2_a, y2_b);
      
          float inter_w = std::max(0.0f, x2 - x1);
          float inter_h = std::max(0.0f, y2 - y1);
          float inter_area = inter_w * inter_h;
      
          float area_a = (x2_a - x1_a) * (y2_a - y1_a);
          float area_b = (x2_b - x1_b) * (y2_b - y1_b);
          float union_area = area_a + area_b - inter_area;
      
          return union_area > 0 ? inter_area / union_area : 0;
      }
      
      // Non-Maximum Suppression for YOLO26 detections
      static std::vector<yolo26_detection_t> yolo26_nms(std::vector<yolo26_detection_t> &detections, float nms_threshold)
      {
          // Sort by confidence (descending)
          std::sort(detections.begin(), detections.end(),
                    [](const yolo26_detection_t &a, const yolo26_detection_t &b)
                    { return a.confidence > b.confidence; });
      
          std::vector<yolo26_detection_t> result;
          std::vector<bool> suppressed(detections.size(), false);
      
          for (size_t i = 0; i < detections.size(); i++)
          {
              if (suppressed[i])
                  continue;
      
              result.push_back(detections[i]);
      
              for (size_t j = i + 1; j < detections.size(); j++)
              {
                  if (suppressed[j])
                      continue;
      
                  // Only suppress same class
                  if (detections[i].class_id != detections[j].class_id)
                      continue;
      
                  float iou = calculate_iou(
                      detections[i].x1, detections[i].y1, detections[i].x2, detections[i].y2,
                      detections[j].x1, detections[j].y1, detections[j].x2, detections[j].y2);
      
                  if (iou > nms_threshold)
                  {
                      suppressed[j] = true;
                  }
              }
          }
      
          return result;
      }
      
      // Parse YOLO26 2-output format: boxes [1, 4, N] + scores [1, nc, N]
      static std::vector<yolo26_detection_t> parse_yolo26_2output(
          rknn_tensor_mem **output_mems,
          rknn_tensor_attr *output_attrs,
          int model_width, int model_height,
          float conf_threshold, int num_classes,
          bool boxes_transposed)
      {
          std::vector<yolo26_detection_t> detections;
      
          int8_t *boxes_data = (int8_t *)output_mems[0]->virt_addr;
          int32_t boxes_zp = output_attrs[0].zp;
          float boxes_scale = output_attrs[0].scale;
      
          int8_t *scores_data = (int8_t *)output_mems[1]->virt_addr;
          int32_t scores_zp = output_attrs[1].zp;
          float scores_scale = output_attrs[1].scale;
      
          // Determine number of anchors
          int num_anchors;
          if (boxes_transposed)
          {
              num_anchors = output_attrs[0].dims[2];
          }
          else
          {
              num_anchors = output_attrs[0].dims[1];
          }
      
          // Check if scores are already sigmoid or logits
          float scores_min = 9999, scores_max = -9999;
          for (int i = 0; i < (int)output_attrs[1].n_elems; i++)
          {
              float val = dequantize(scores_data[i], scores_zp, scores_scale);
              if (val < scores_min)
                  scores_min = val;
              if (val > scores_max)
                  scores_max = val;
          }
          bool scores_are_sigmoid = (scores_min >= -0.1f && scores_max <= 1.1f);
      
          for (int i = 0; i < num_anchors; i++)
          {
              // Get box coordinates
              float x1, y1, x2, y2;
              if (boxes_transposed)
              {
                  // [1, 4, N] - channel first
                  x1 = dequantize(boxes_data[0 * num_anchors + i], boxes_zp, boxes_scale);
                  y1 = dequantize(boxes_data[1 * num_anchors + i], boxes_zp, boxes_scale);
                  x2 = dequantize(boxes_data[2 * num_anchors + i], boxes_zp, boxes_scale);
                  y2 = dequantize(boxes_data[3 * num_anchors + i], boxes_zp, boxes_scale);
              }
              else
              {
                  // [1, N, 4] - anchor first
                  x1 = dequantize(boxes_data[i * 4 + 0], boxes_zp, boxes_scale);
                  y1 = dequantize(boxes_data[i * 4 + 1], boxes_zp, boxes_scale);
                  x2 = dequantize(boxes_data[i * 4 + 2], boxes_zp, boxes_scale);
                  y2 = dequantize(boxes_data[i * 4 + 3], boxes_zp, boxes_scale);
              }
      
              // Check if format is xywh instead of xyxy
              if (x2 < x1 || y2 < y1)
              {
                  // Assume xywh format: convert to xyxy
                  float cx = x1, cy = y1, w = x2, h = y2;
                  x1 = cx - w / 2.0f;
                  y1 = cy - h / 2.0f;
                  x2 = cx + w / 2.0f;
                  y2 = cy + h / 2.0f;
              }
      
              // Find max class score
              float max_score = -9999;
              int max_class = 0;
              for (int c = 0; c < num_classes; c++)
              {
                  float score;
                  if (boxes_transposed)
                  {
                      // [1, nc, N]
                      score = dequantize(scores_data[c * num_anchors + i], scores_zp, scores_scale);
                  }
                  else
                  {
                      // [1, N, nc]
                      score = dequantize(scores_data[i * num_classes + c], scores_zp, scores_scale);
                  }
      
                  if (!scores_are_sigmoid)
                  {
                      score = sigmoid(score);
                  }
      
                  if (score > max_score)
                  {
                      max_score = score;
                      max_class = c;
                  }
              }
      
              if (max_score < conf_threshold)
                  continue;
      
              // Clamp to model bounds
              x1 = std::max(0.0f, std::min(x1, (float)model_width));
              y1 = std::max(0.0f, std::min(y1, (float)model_height));
              x2 = std::max(0.0f, std::min(x2, (float)model_width));
              y2 = std::max(0.0f, std::min(y2, (float)model_height));
      
              // Skip invalid boxes
              if (x2 <= x1 || y2 <= y1)
                  continue;
      
              yolo26_detection_t det;
              det.x1 = x1;
              det.y1 = y1;
              det.x2 = x2;
              det.y2 = y2;
              det.confidence = max_score;
              det.class_id = max_class;
      
              detections.push_back(det);
          }
      
          return detections;
      }
      
      int yolo26_init_postprocess(rknn_app_context_t *app_ctx, yolo26_context_t *yolo26_ctx)
      {
          if (!app_ctx || !yolo26_ctx)
          {
              printf("[YOLO26] ERROR: Invalid parameters\n");
              return -1;
          }
      
          memset(yolo26_ctx, 0, sizeof(yolo26_context_t));
      
          int n_output = app_ctx->io_num.n_output;
      
          printf("[YOLO26] Initializing 2-output format (%d outputs)...\n", n_output);
      
          // Print output tensor info
          for (int i = 0; i < n_output; i++)
          {
              printf("  Output %d: dims=[%d,%d,%d,%d], size=%d\n", i,
                     app_ctx->output_attrs[i].dims[0],
                     app_ctx->output_attrs[i].dims[1],
                     app_ctx->output_attrs[i].dims[2],
                     app_ctx->output_attrs[i].dims[3],
                     app_ctx->output_attrs[i].size);
          }
      
          if (n_output != 2)
          {
              printf("[YOLO26] ERROR: Expected 2 outputs (boxes + scores), got %d\n", n_output);
              return -1;
          }
      
          // Parse 2-output format: boxes + scores
          int boxes_dim1 = app_ctx->output_attrs[0].dims[1];
          int boxes_dim2 = app_ctx->output_attrs[0].dims[2];
          int scores_dim1 = app_ctx->output_attrs[1].dims[1];
          int scores_dim2 = app_ctx->output_attrs[1].dims[2];
      
          if (boxes_dim1 == 4)
          {
              // [1, 4, N] boxes format
              yolo26_ctx->num_predictions = boxes_dim2;
              yolo26_ctx->boxes_transposed = true;
              yolo26_ctx->num_classes = scores_dim1;
              printf("[YOLO26] Format: boxes [1,4,%d], scores [1,%d,%d]\n",
                     boxes_dim2, scores_dim1, scores_dim2);
          }
          else if (boxes_dim2 == 4)
          {
              // [1, N, 4] boxes format
              yolo26_ctx->num_predictions = boxes_dim1;
              yolo26_ctx->boxes_transposed = false;
              yolo26_ctx->num_classes = scores_dim2;
              printf("[YOLO26] Format: boxes [1,%d,4], scores [1,%d,%d]\n",
                     boxes_dim1, scores_dim1, scores_dim2);
          }
          else
          {
              printf("[YOLO26] ERROR: Invalid boxes dimensions [1,%d,%d], expected [1,4,N] or [1,N,4]\n",
                     boxes_dim1, boxes_dim2);
              return -1;
          }
      
          printf("[YOLO26] Predictions: %d, Classes: %d\n",
                 yolo26_ctx->num_predictions, yolo26_ctx->num_classes);
      
          return 0;
      }
      
      void yolo26_deinit_postprocess()
      {
          // Nothing to cleanup currently
      }
      
      int yolo26_post_process(rknn_app_context_t *app_ctx, yolo26_context_t *yolo26_ctx,
                              void *outputs, letterbox_t *letter_box,
                              float conf_threshold, float nms_threshold,
                              object_detect_result_list *od_results)
      {
          if (!app_ctx || !yolo26_ctx || !outputs || !letter_box || !od_results)
          {
              return -1;
          }
      
          rknn_tensor_mem **output_mems = (rknn_tensor_mem **)outputs;
      
          int model_width = app_ctx->model_width;
          int model_height = app_ctx->model_height;
      
          // Parse 2-output format
          std::vector<yolo26_detection_t> raw_detections = parse_yolo26_2output(
              output_mems,
              app_ctx->output_attrs,
              model_width, model_height,
              conf_threshold,
              yolo26_ctx->num_classes > 0 ? yolo26_ctx->num_classes : 1,
              yolo26_ctx->boxes_transposed);
      
          // Apply NMS
          std::vector<yolo26_detection_t> final_detections = yolo26_nms(raw_detections, nms_threshold);
      
          // Convert to output format with letterbox coordinate mapping
          memset(od_results, 0, sizeof(object_detect_result_list));
          int count = 0;
      
          for (const auto &det : final_detections)
          {
              if (count >= OBJ_NUMB_MAX_SIZE)
                  break;
      
              // Map coordinates back to original image space
              float x1 = det.x1 - letter_box->x_pad;
              float y1 = det.y1 - letter_box->y_pad;
              float x2 = det.x2 - letter_box->x_pad;
              float y2 = det.y2 - letter_box->y_pad;
      
              // Scale back to original coordinates
              od_results->results[count].box.left = (int)(clamp(x1, 0, model_width) / letter_box->scale);
              od_results->results[count].box.top = (int)(clamp(y1, 0, model_height) / letter_box->scale);
              od_results->results[count].box.right = (int)(clamp(x2, 0, model_width) / letter_box->scale);
              od_results->results[count].box.bottom = (int)(clamp(y2, 0, model_height) / letter_box->scale);
              od_results->results[count].prop = det.confidence;
              od_results->results[count].cls_id = det.class_id;
      
              count++;
          }
      
          od_results->count = count;
      
          return 0;
      }

      And you're good to go! 🚀

    3. Glenn JocherUltralytics(已编辑)

      Thanks for sharing this—splitting the head into 2 outputs (boxes + scores) is a sensible RV1106 workaround to keep RKNN-friendly ops and avoid INT8 scale/zero-point coupling issues; just note that monkey-patching head.Detect.forward is inherently version-fragile. Current Ultralytics already provides an INT8 flow for RV1106 via quantize=8 with data= calibration images (see the Rockchip RKNN export guide), so please try that first. If you still need your custom split, open a GitHub issue linking back to this thread and include !yolo checks, your rknn-toolkit2 version, and a minimal calibration set description so we can evaluate it.

    1. Glenn JocherUltralytics

      @Arshapjoy thanks for the detailed info; for YOLO11 detection heads the export output shape is (1, nc + 4, 8400) (4 box coords plus nc class scores), so with 2 classes (1, 6, 8400) is expected and both classes are present, whereas a postprocessor assuming 5 + nc (old YOLO layout with an extra objectness channel) will effectively drop one class. I’d suggest updating your RKNN-side decoder to treat channels [0:4] as [x, y, w, h] and [4:] as class logits for all nc classes; if you still see a true (1, 7, 8400) PyTorch output on the same checkpoint, please share your ultralytics version and yolo checks output so we can dig deeper. The overall YOLO11 head behaviour is also outlined in the YOLO11 model documentation.

      @BakhtiariB the high-level YOLO("./yolo11n_rknn_model") interface is designed to keep preprocessing, NPU inference and postprocessing together, so if you need to run only the core RKNN inference on the NPU while fully controlling pre/post on the CPU you’ll need to load the exported .rknn with Rockchip’s RKNN API directly and implement your own preprocessing and YOLO11 postprocessing around it.

      @dttvn0010 the ~99 ms figure is a conservative end-to-end pipeline measurement on our RK3588 test setup, so seeing ~11 ms from a highly tuned RKNN deployment using all three NPU cores is quite reasonable; if you are open to sharing your exact model variant, quantization mode, input size and NPU core configuration, we can use that to cross-check and better explain the gap between baseline and fully optimized numbers.

      @NikolayKash in practice, large gains like that usually come from combining INT8-quantized RKNN models, all three NPU cores enabled, a suitable input resolution and very lean Python-side pre/postprocessing, so it would be great if @dttvn0010 can share their exact script and RK3588 configuration for others to replicate.

  5. guojieliGitHub

    he YOLO11 benchmark was run by the Ultralytics team on a Radxa Rock 5B based on the Rockchip RK3588, using the RKNN model format to evaluate speed and accuracy. Could you please clarify whether this was an INT8-quantized model?

  6. ArshapjoyGitHub

    Exported RKNN/ONNX model only has 1 output class instead of 2


    Overview

    I’m using Ultralytics YOLOv11s with two custom classes and exporting it to ONNX and RKNN formats. After conversion, inference on the RKNN device only detects a single class, and the output tensor shape is (1, 6, 8400) instead of the expected (1, 7, 8400). I’ve validated that the source .pt model correctly reports 2 classes, but both ONNX and RKNN exports drop one class.


    Model Details

    • Checkpoint path: path/to/yolo11s_custom.pt
    • Number of classes: 2
    • Input size: 640×640

    from ultralytics import YOLO

    Load the YOLO11 model

    model = YOLO("custom_yolo11s.pt")

    Export the model to RKNN format

    rknn_path = model.export(format="rknn", name="rk3588") print(f"RKNN model saved to: {rknn_path}")

    Actual Behavior

    • Received ONNX , RKNN and metadata.yaml after conversion
    • Exported ONNX model output shape: (1, 6, 8400)
    • Exported RKNN model output shape: (1, 6, 8400)
    • Only one class is detected during inference on the RK3588 board.

    7. Logs & Metadata

    • metadata.yaml generated alongside the RKNN model:
      model:
        name: yolo11s_custom
        format: rknn
        input_shape: [1, 3, 640, 640]
        output_shape: [1, 6, 8400]
      classes: 2
    • ONNX graph info: Attached minimal onnx.shape_inference log shows only 6 channels in the final layer.

    8. Additional Notes

    • Manual ONNX export yields the same issue, so it’s unlikely to be an Ultralytics export bug alone.
    • I’ve confirmed that the PyTorch forward pass on the .pt checkpoint produces a shape (1, 7, 8400).

    9. Request Could you please:

    1. Confirm whether there’s an option/flag in the YOLO-Ultralytics or RKNN export pipeline I’m missing to preserve all classes.
    2. Suggest any workarounds or patches to ensure correct output-channel dimensionality.
    3. Identify if this might be a known issue in RKNN-Toolkit 1.5.0 or Ultralytics YOLO v11.

    Thank you for your assistance!

  7. WeiIIzzGitHub

    Hello, when I exported the rknn model, I performed int8 quantization, but I had an error like this: E build: If set do_quantization = True, dataset file need be provided. But it`s value is None! E export_rknn: RKNN model does not exist, please load build model first! My export code is: from ultralytics import YOLO

    model = YOLO("/home/ky/dfxw/runs/detect/train3/weights/best.pt") model.export(format="rknn", name="rk3576", int8=True, data="/home/ky/rknn_yolo_test/data.yaml") The path pointed to by data is the same as the training dataset. How to solve it

    1. WeiIIzzGitHub

      The content of data.yaml is as follows path: /home/ky/dfxw/split_jiaqian # dataset root dir train: images/train val: images/val

      names: 0: 0 1: 1

    2. Glenn JocherUltralytics(已编辑)

      @WeiIIzz INT8 export for RKNN is supported with quantize=8 (it replaces the deprecated int8=True) together with data pointing to a dataset YAML, which Ultralytics converts into the calibration image list that RKNN needs:

      from ultralytics import YOLO
      
      model = YOLO("/home/ky/dfxw/runs/detect/train3/weights/best.pt")
      model.export(format="rknn", name="rk3576", quantize=8, data="/home/ky/rknn_yolo_test/data.yaml")

      The error you saw comes from an older version that did not pass a calibration dataset to RKNN, so run pip install -U ultralytics and retry. Make sure the YAML points to existing images. Let us know if this resolves your issue!

    3. WeiIIzzGitHub

      @glenn-jocher I followed your steps to modify the code as follows. But I still encountered the same problem.

      from ultralytics import YOLO
      
      model = YOLO("/home/ky/dfxw/runs/detect/train3/weights/best.pt")
      model.export(format="rknn", name="rk3576", int8=True, data="/home/ky/rknn_yolo_test/images")

      Is there only images in the data path?

    4. Glenn JocherUltralytics(已编辑)

      @WeiIIzz older releases ignored int8=True for RKNN and built the model without quantization. Current Ultralytics supports INT8 RKNN export, so after pip install -U ultralytics you can export with:

      model.export(format="rknn", name="rk3576", quantize=8, data="data.yaml")

      Here data is used for the calibration images. Without quantize, the export builds a floating-point (FP16) model.

  8. VictorAbobaGitHub

    Good day,

    Is RKNN available for obb tasks? I tried to use it directly, but got an error saying that some operations (cos) are not available in RKNN. As a fix, RKNN suggested adding disable_rules = ['reduce_reshape_op_around_concat'] to my configuration. I actually did this inside the YOLO library. Model was exported first to ONNX and then from ONNX to RKNN works and returns the responses in the format (1,6,21504), which seems what I expected. However, if I use this model through YOLO ("path_to_rknn_model", task = 'obb'), will I still get the results in the standard YOLO format? I just don't want to prescribe pre - and post-processing on my own :)

    Thank you!

    1. Glenn JocherUltralytics(已编辑)

      @VictorAboba thank you for your question! RKNN export now supports all seven Ultralytics tasks, including OBB, so you can export OBB models directly with format="rknn" and load the exported _rknn_model directory with YOLO(...) to get standard YOLO results without writing your own pre- and post-processing. See the Rockchip RKNN guide for details.

    2. SpidrreGitHub

      Hey! Did you notice any significant decrease in quality of your model after conversion? I am also trying to find a way to convert from OBB onnx to RKNN without 'reduce_reshape_op_around_concat'.

    3. VictorAbobaGitHub

      Hey! Did you notice any significant decrease in quality of your model after conversion? I am also trying to find a way to convert from OBB onnx to RKNN without 'reduce_reshape_op_around_concat'.

      Hello, after @glenn-jocher answer, I didn't attempt to convert the OBB model. We have a few other models, and all of them except YOLO were converted to the RKNN format. We will try using this stuff (YOLO without an NPU and others with it) ^_^ After conversion, the OCR model, for example, has almost the same quality, but it doesn't need 'reduce_reshape_op_around_concat' to be used.