Get Started

Rockchip RKNN Export

RKNN is the model format of the NPUs in Rockchip processors such as the RK3588 and RK3566. Exporting an Ultralytics YOLO26 model with format="rknn" creates a yolo26n_rknn_model/ directory that holds a .rknn file built for one Rockchip chip, which you select with the name argument, in FP16 or INT8 precision.

At a Glance#

PropertyDetails
format argumentrknn
Outputyolo26n_rknn_model/, a directory with yolo26n-rk3588.rknn and metadata.yaml; INT8 exports of detect, segment, pose, and OBB models add tail.onnx
Runs onThe NPU of one Rockchip chip per export: RK3588, RK3576, RK3566, RK3568, RK3562, RK2118, RV1126B, RV1103, RV1106, RV1103B, or RV1106B, see Target Chips
RuntimeRKNN Toolkit Lite2 (the rknn-toolkit-lite2 Python package) on the Rockchip device
PrecisionFP16 (default) and INT8; INT8 only on RV1103, RV1106, RV1103B, and RV1106B. There is no FP32 export. YOLO26n for the RK3588 is 7.4 MB in FP16 and 4.3 MB in INT8
Export hostLinux x86-64 with Python 3.12 or older. Export on a PC and copy the exported directory to the Rockchip device
Export in the cloud

Ultralytics Platform can export a trained model to RKNN for the Rockchip chip you select, so you do not need a local export environment.

Quickstart#

Install the Ultralytics package in a separate virtual environment on an x86-64 Linux machine, together with the protobuf and numpy versions that the RKNN toolkit needs:

pip install ultralytics "protobuf<7" "numpy<2.4"

The first export installs onnx, onnxslim, onnxruntime, and rknn-toolkit2 with its dependencies if they are missing. With a plain pip install ultralytics, the first export also replaces numpy, protobuf, and setuptools when their versions are outside the ranges below, so use a separate environment. When it has to replace protobuf or numpy, that export fails, and the next one in a new process passes. The export has these version requirements:

PackageRequired version
Python3.12 or older, the newest version that rknn-toolkit2 2.3.2 has builds for
rknn-toolkit22.3.2, the only release on PyPI in the required range (2.3.2 or newer and below 2.5)
onnx1.16.1 or newer and below 1.19.0
numpybelow 2.4
protobufbelow 7
setuptoolsbelow 82

For other installation methods, see the installation guide. Then export your model on the PC, copy the exported directory to the Rockchip device, and load it there to run inference or validate its accuracy.

Export
from ultralytics import YOLO

# Load a YOLO26 model
model = YOLO("yolo26n.pt")

# Export the model to RKNN format for the RK3588 in FP16
model.export(format="rknn", name="rk3588")  # creates 'yolo26n_rknn_model/'

# Or export an INT8-quantized RKNN model with calibration data instead
model.export(format="rknn", name="rk3588", quantize=8, data="coco8.yaml")  # creates 'yolo26n_rknn_model/'

Both exports write the same yolo26n_rknn_model/ directory, so the second one replaces the first. Predict and Validate run only on a Rockchip device:

Predict
from ultralytics import YOLO

# Load the exported RKNN model on the Rockchip device
model = YOLO("yolo26n_rknn_model")

# Run inference
results = model("https://ultralytics.com/images/bus.jpg")
Validate
from ultralytics import YOLO

# Load the exported RKNN model on the Rockchip device
model = YOLO("yolo26n_rknn_model")

# Validate accuracy on the COCO8 dataset
metrics = model.val(data="coco8.yaml")

Supported Tasks#

RKNN export supports all seven Ultralytics tasks with YOLOv8, YOLO11, and YOLO26 models: detection, segmentation, semantic segmentation, depth estimation, classification, pose estimation, and OBB. Semantic segmentation and depth estimation are available only with YOLO26, the only family that ships those heads.

Export Arguments#

ArgumentTypeDefaultDescription
formatstr'rknn'Target format for the exported model, defining compatibility with various deployment environments.
namestr'rk3588'Rockchip chip the model is built for, in upper or lower case. See Target Chips.
imgszint or tuple640Image size of the model input: an integer for square images or a tuple (height, width). When not passed, the export uses the size the checkpoint was trained at. The official YOLO26 checkpoints use 640 for detect, segment, and pose models and for semantic models pretrained on ADE20K, 1024 for OBB models and for semantic models pretrained on Cityscapes, 768 for depth, and 224 for classify.
quantizeint or strNonePrecision: 16 or unset builds FP16, and 8 builds INT8 calibrated on data. For the INT8-only chips, unset builds INT8 and 16 raises an error. 32 raises an error, and a checkpoint trained with quantize=8 cannot be exported to RKNN, see Limitations and Troubleshooting. Replaces the deprecated half/int8 flags.
datastrNoneDataset YAML used for INT8 calibration; classification instead takes a dataset directory or a built-in dataset name. Calibration uses the val split unless split is set. If omitted with quantize=8, Ultralytics uses the small default dataset for the model task, such as coco8.yaml for detection.
fractionfloat, int, or list1.0Calibration subset: a ratio, an image count (integers above 1), or a [train, val, test] list, from which the entry of the calibration split is used.
batchint1Batch size of the exported model, fixed in the input shape.
simplifyboolTrueSimplifies the intermediate ONNX graph with onnxslim.
opsetintNoneONNX opset version of the intermediate ONNX graph. Uses 19 when unset, and reduces higher values to 19.
nmsbool, optionalNoneRKNN models always return raw predictions for external NMS. The NMS-free head (False) and embedded NMS (True) are not available, see Limitations and Troubleshooting.
devicestrNoneDevice that traces the intermediate ONNX graph: cpu (the default) or 0 for a CUDA GPU.

For the arguments shared by all formats, see the Export mode documentation.

Target Chips#

A .rknn file is built for one chip, and name selects it. The sizes are for a YOLO26n detection model exported at imgsz=640 with quantize unset:

namePrecisionsUnset quantize buildsYOLO26n fileSize
rk3588FP16, INT8FP16yolo26n-rk3588.rknn7.4 MB
rk3576FP16, INT8FP16yolo26n-rk3576.rknn10.5 MB
rk3566FP16, INT8FP16yolo26n-rk3566.rknn6.8 MB
rk3568FP16, INT8FP16yolo26n-rk3568.rknn6.8 MB
rk3562FP16, INT8FP16yolo26n-rk3562.rknn6.8 MB
rk2118FP16, INT8FP16yolo26n-rk2118.rknn6.8 MB
rv1126bFP16, INT8FP16yolo26n-rv1126b.rknn7.1 MB
rv1103INT8INT8yolo26n-rv1103.rknn4.2 MB
rv1106INT8INT8yolo26n-rv1106.rknn4.2 MB
rv1103bINT8INT8yolo26n-rv1103b.rknn4.2 MB
rv1106bINT8INT8yolo26n-rv1106b.rknn4.2 MB

The four INT8-only chips are calibrated on the default coco8.yaml in this table, and their exports also write tail.onnx.

Quantization#

quantize selects the precision of the .rknn file. The sizes are for an export at imgsz=640 and batch=1:

quantizePrecisionCalibration dataFiles for YOLO26n and rk3588Size
unset or 16FP16Noyolo26n-rk3588.rknn7.4 MB
8INT8Yesyolo26n-rk3588.rknn and tail.onnx4.3 MB and 0.2 MB
  • FP16: the default for the seven chips that support it. It needs no calibration data.
  • INT8: quantize=8 runs calibration images through the model and stores weights and activations as 8-bit integers. The export calibrates on the val split of data, or on the split you set with split, and warns below 300 images. Use fraction to limit the number of images.
  • Mixed precision: on the seven chips that support FP16, an INT8 export lets RKNN Toolkit keep layers in FP16 when quantization changes their output too much (auto_hybrid). The four INT8-only chips skip this step.
  • Detection head: for detect, segment, pose, and OBB models, the INT8 .rknn file ends at the output convolutions of the head. The operations that decode those outputs into boxes are saved as tail.onnx and run on the CPU of the device in floating point, because per-tensor INT8 cannot hold the pixel-range values of the box decoding.
  • Depth models: depth INT8 exports calibrate on the images of a depth dataset that have paired depth maps. Choose calibration images that span the near and far depth range of the deployment scene. With the default depth8.yaml, which has 4 validation images, a YOLO26n-depth INT8 model in the rknn-toolkit2 simulator returned depths up to 3.9 for a street image on which the FP16 model returned up to 13.9.
  • On-device status: INT8 exports of every task were run in the rknn-toolkit2 simulator and compared with the ONNX model on a single image. Their accuracy has not been measured on a dataset or on Rockchip hardware, so validate an INT8 model on your device and your own data before you deploy it.
Example
from ultralytics import YOLO

model = YOLO("yolo26n.pt")

# Calibrate on 10% of the COCO validation images (500 images); replace coco.yaml with your own dataset
model.export(format="rknn", name="rk3588", quantize=8, data="coco.yaml", fraction=0.1)

Output Format#

The export writes one directory:

FileContents
yolo26n-rk3588.rknnThe model, with the chip set by name in the file name
tail.onnxOnly for INT8 detect, segment, pose, and OBB models: the operations that decode the head outputs into raw predictions, to run on the CPU
metadata.yamlClass names, task, image size, batch size, stride, and export arguments, which Ultralytics reads when it loads the model

The intermediate yolo26n.onnx file stays next to the directory. The .rknn model takes one input with shape (batch, height, width, 3): RGB, channels last (NHWC), uint8 with values from 0 to 255. The division by 255 is part of the model, so do not scale the image yourself. This differs from the ONNX export, which takes channels-first float32 images scaled to 0-1.

The outputs depend on the task and the precision. The shapes below are for the official YOLO26n checkpoints at their default export size, with the Cityscapes checkpoint yolo26n-sem.pt for semantic segmentation:

TaskDefault imgszFP16 output, and tail.onnx output for INT8INT8 .rknn output
Detect640(1, 84, 8400)6 head tensors, decoded by tail.onnx
Segment640(1, 116, 8400) and (1, 32, 160, 160)9 head tensors and (1, 32, 160, 160), decoded by tail.onnx
Semantic1024(1, 19, 1024, 1024), a score for each class and pixelSame as FP16
Depth768(1, 1, 768, 768), depth mapSame as FP16
Classify224(1, 1000), class probabilitiesSame as FP16
Pose640(1, 56, 8400)9 head tensors, decoded by tail.onnx
OBB1024(1, 20, 21504)9 head tensors, decoded by tail.onnx
  • Raw predictions: FP16 models, and tail.onnx for INT8 models, return the same raw predictions as a default ONNX export: for each anchor the box center x, center y, width, and height in input pixels, then the class scores, then the values of the task (32 mask coefficients, 17 x 3 keypoint values, or the OBB angle in radians). Apply NMS yourself. See the ONNX output format for the layout of each task.
  • INT8 head tensors: an INT8 detection model returns two tensors for each of the three detection scales, one with box values and one with class scores, and segmentation, pose, and OBB models return one more tensor per scale with their extra values. Feed them to tail.onnx in the order the model returns them; a segmentation model also passes its mask prototypes through tail.onnx unchanged.
  • Semantic segmentation: the output holds a score for each class and pixel, not the class map of the ONNX export, so take the argmax over the class axis in your own code.
  • NMS: there is no NMS-free or embedded-NMS output, see Limitations and Troubleshooting.

YOLO("yolo26n_rknn_model") reads the layout from metadata.yaml, runs tail.onnx, and applies NMS, so you only need these details when you run the model in your own code. The keys of metadata.yaml include names, task, imgsz, stride, batch, end2end, args, and, for pose models, kpt_shape.

Run with RKNN Toolkit Lite2#

To run the model on the device outside the Ultralytics package, load it with the rknn-toolkit-lite2 package and, for an INT8 detect, segment, pose, or OBB model, pass its outputs to tail.onnx with ONNX Runtime:

# Export first: yolo export model=yolo26n.pt format=rknn name=rk3588 quantize=8 data=coco8.yaml
import numpy as np
import onnxruntime as ort
from rknnlite.api import RKNNLite

rknn = RKNNLite()
assert rknn.load_rknn("yolo26n_rknn_model/yolo26n-rk3588.rknn") == 0
assert rknn.init_runtime() == 0

image = np.zeros((1, 640, 640, 3), dtype=np.uint8)  # your letterboxed image: RGB, NHWC, 0-255
outputs = rknn.inference(inputs=[image])  # six head tensors for an INT8 detection model

tail = ort.InferenceSession("yolo26n_rknn_model/tail.onnx", providers=["CPUExecutionProvider"])
feed = {t.name: x.astype(np.float32) for t, x in zip(tail.get_inputs(), outputs)}
predictions = tail.run(None, feed)[0]  # shape (1, 84, 8400), raw predictions that need NMS
assert predictions.shape == (1, 84, 8400), "export a detection model at imgsz=640 with quantize=8"

tail.onnx has a batch size of 1 whatever the export batch, so run it once per image for a model exported with a larger batch. An FP16 model has no tail.onnx, and outputs[0] already holds the raw predictions.

Deploy#

On the Rockchip device, install the package with pip install ultralytics and copy the yolo26n_rknn_model/ directory to it; the protobuf and numpy versions from the Quickstart are needed only for the export. YOLO("yolo26n_rknn_model") then runs the model on the NPU with the rknn-toolkit-lite2 Python package, which Ultralytics installs on first use together with onnxruntime when the directory holds a tail.onnx. The device needs Python 3.12 or older, the newest version that rknn-toolkit-lite2 has builds for.

  • Rockchip device required: Ultralytics loads an RKNN model only on 64-bit ARM Linux systems whose device tree compatible string ends with one of the supported chip names. Everywhere else, loading fails, see Limitations and Troubleshooting. rknn-toolkit-lite2 also has builds for 64-bit ARM Linux only, so on a target that does not meet these conditions, run the .rknn file with Rockchip's C API and either run tail.onnx or implement the same decoding in your own code.
  • Matching chip: export with the name of the chip on your board.
  • Fixed input shape: the model always runs at its export imgsz and batch, which Ultralytics reads from metadata.yaml. An imgsz passed to predict or val is ignored. Larger batches are split into chunks of the export batch size and smaller ones are padded, so any number of images works.
  • device: there is no device to select. The .rknn model runs through the RKNN runtime, and tail.onnx runs with ONNX Runtime on the CPU.

An earlier version of this workflow was tested on a Radxa Rock 5B (RK3588) and a Radxa Zero 3W (RK3566). The current INT8 export has not yet been run on hardware.

Deployment Guides#

Limitations and Troubleshooting#

  • Export host: export on an x86-64 Linux machine. On macOS the export cannot install rknn-toolkit2 and fails with No module named 'rknn'; the package has no macOS or Windows builds. Export on ARM64 Linux, including Rockchip boards, is untested and not supported: Ultralytics tests the RKNN export only on x86-64 Linux.

  • Python version: rknn-toolkit2 2.3.2 has builds for Python 3.12 and older. On a newer Python, create the environment with Python 3.12.

  • First export in a new environment: in an environment without the protobuf and numpy versions from the Quickstart, the first export replaces these packages during the run and fails. When it replaces protobuf 7, which pip installs together with onnx, the error is AttributeError: 'google._upb._message.FieldDescriptor' object has no attribute 'label'; when it replaces only numpy, the error is TypeError: only 0-dimensional arrays can be converted to Python scalars. Run the export again in a new process, which means a kernel restart in a notebook, or install "protobuf<7" "numpy<2.4" before the first export as in the Quickstart.

  • RKNN models do not run on a PC: predicting or validating with YOLO("yolo26n_rknn_model") fails with RKNN inference is only supported on Rockchip devices. on a PC, and on a board whose /proc/device-tree/compatible string does not end with one of the supported chip names. Print it with cat /proc/device-tree/compatible.

  • FP32: quantize=32 fails with quantize=32 (FP32) is not supported; format='rknn' supports quantize=16 (FP16), 8 (INT8).

  • INT8-only chips: for rv1103, rv1106, rv1103b, and rv1106b, an export without quantize logs Rockchip target 'rv1106' requires INT8 quantization, enabling it. and builds an INT8 model, and quantize=16 fails with Rockchip target 'rv1106' only supports INT8, but got quantize=16.

  • Missing or wrong name: without name, the export logs Rockchip RKNN export requires a missing 'name' arg for processor type. Using default name='rk3588'. An unknown value fails with Invalid processor name 'rk9999' for Rockchip RKNN export. followed by the valid names.

  • NMS-free head: RKNN models always return raw predictions that need NMS. With nms=False, the export of a YOLO26 or YOLOv10 model logs This export format does not support end2end models, disabling the end2end branch. and uses the one-to-many head; for other detection, segmentation, pose, and OBB models, for example YOLO11, it logs This model has no one-to-one head; using one-to-many outputs. Classification, semantic segmentation, and depth models ignore nms=False.

  • Embedded NMS: nms=True logs format=rknn does not support embedded NMS; exporting native outputs for external NMS. and writes the raw output.

  • Checkpoints trained with quantize=8: a checkpoint fine-tuned with quantization-aware training fails with format='rknn' cannot export a QAT model: its Q/DQ ranges are only read by the 'onnx' and 'engine' backends. Export the checkpoint you started the fine-tuning from with quantize=8 instead.

  • RT-DETR: with rknn-toolkit2 2.3.2, the export of an RT-DETR model fails inside the toolkit in FP16 and in INT8, for rtdetr-l.pt at imgsz=640 with ValueError: operands could not be broadcast together with shapes (256,256) (8400,1).

  • Fixed input shape: dynamic=True fails with argument 'dynamic' is not supported for format='rknn'. Export again to change the image size or the batch size.

  • Calibration without data: a quantize=8 export of a detection model logs INT8 export requires a missing 'data' arg for calibration. Using default 'data=coco8.yaml'. and calibrates on that small dataset; the other tasks name their own default dataset. Pass data with images that represent your deployment data.

  • Calibration size: the export logs >300 images recommended for INT8 calibration, found 4 images. when the calibration split has fewer than 300 images. coco8.yaml has only 4 validation images, which is enough to test the export but not to calibrate a model for deployment.

  • Output directory: every export of one checkpoint writes the same yolo26n_rknn_model/ directory, whatever its name, quantize, imgsz, or batch, and deletes the previous contents first. Rename the directory before you export for a second chip or precision, and keep _rknn_model in its name, for example yolo26n_rk3566_rknn_model/, because Ultralytics recognizes the format by it.

  • ONNX opset: an opset above 19 logs rknn-toolkit2 requires opset<=19, setting opset=19.

  • INT8 models from earlier versions: re-export INT8 detection, segmentation, pose, and OBB models that were exported before ultralytics 8.4.191, the release that added tail.onnx. Those models return box and keypoint coordinates relative to the input size, which current releases no longer rescale, so they load without an error and return wrong boxes.

  • Runtime version mismatch: if the log on the device reports that the RKNN runtime version does not match the RKNN Toolkit version and inference fails, replace /usr/lib/librknnrt.so with the official librknnrt.so file.

    RKNN log warning that the model toolkit version does not match the librknnrt runtime version

For other errors, see the Common Issues guide.

Benchmarks#

YOLO26 benchmarks below were run by the Ultralytics team on Radxa Rock 5B based on Rockchip RK3588 with rknn model format measuring speed and accuracy.

Performance
ModelFormatPrecisionStatusSize (MB)mAP50-95(B)Inference time (ms/im)
YOLO26nrknnFP16✅7.20.47758.5
YOLO26nrknnINT8✅4.10.46341.2
YOLO26srknnFP16✅21.00.56993.8
YOLO26srknnINT8✅12.00.55461.9
YOLO26mrknnFP16✅43.00.608226.3
YOLO26mrknnINT8✅22.00.603122.0
YOLO26lrknnFP16✅53.00.628264.7
YOLO26lrknnINT8✅28.00.617145.3
YOLO26xrknnFP16✅113.00.664655.4
YOLO26xrknnINT8✅58.00.649287.4

Benchmarked with ultralytics 8.4.60

Note

Validation for the above benchmarks was done using the COCO128 dataset. Inference time does not include pre/post-processing. The INT8 rows will be refreshed once the current INT8 export path (host-side decode and auto_hybrid) is validated on Rockchip hardware.

The sizes in the table were recorded with that earlier version; the sizes elsewhere on this page are the .rknn file of a current export in MB (10^6 bytes). The YOLO26 model page lists the accuracy of the PyTorch checkpoints on the full COCO validation set.

About RKNN#

Rockchip designs ARM-based systems-on-chip with built-in NPUs, used in single-board computers and embedded products from companies such as Radxa, ASUS, Pine64, Orange Pi, Odroid, Khadas, and Banana Pi. RKNN, short for Rockchip Neural Network, is the model format of those NPUs, and the RKNN Toolkit converts models to it on a PC and runs them on the device.

Rockchip RKNN export for NPU deployment

FAQ#

  • Use the name of the processor on the board, not the name of the board: rk3588 for a Radxa Rock 5B, for example, or rk3566 for a Radxa Zero 3W. On the board, cat /proc/device-tree/compatible prints the string that Ultralytics reads the chip from. A model exported for one chip is not built for another, so export once per chip. See Target Chips for the accepted names.

  • Start with FP16, the default. It needs no calibration data and returns its raw predictions without a second model on the CPU. Choose INT8 (quantize=8) when you need a smaller and faster model or when the chip is INT8-only: YOLO26n for the RK3588 is 4.3 MB in INT8 against 7.4 MB in FP16, and in the benchmarks, which were measured with an earlier export, INT8 ran YOLO26n at 41.2 ms per image against 58.5 ms in FP16. INT8 needs representative calibration data, and for detect, segment, pose, and OBB models it runs the box decoding on the CPU with tail.onnx. Measure latency and accuracy on your device before you choose.

  • Not with YOLO(), which loads RKNN models only on Rockchip devices. To check the model itself on the PC, validate the PyTorch checkpoint or the intermediate yolo26n.onnx file that the export leaves next to the directory, with ONNX Runtime: the RKNN model is converted from that graph. This does not replace a validation on the device, in particular for INT8, because quantization changes the outputs.

  • Yes, with the export format of each accelerator: an RKNN model runs only on Rockchip NPUs. See TensorRT for NVIDIA Jetson, Hailo, Axelera, DEEPX, and Coral Edge TPU, or the model deployment options guide for an overview.

Comments45

Ask a question, share your results, or help someone else.

  1. as-jacksonGitHub

    We are pleased to confirm that the following ArmSoM boards based on Rockchip RK3576 have been successfully tested with Ultralytics projects (e.g., YOLO11n):

    ArmSoM-Sige5 ArmSoM-CM5

    Validation Scope:

    Model training/inference pipeline

    1
  2. yizi72GitHub

    I tried to convert the yolo11n.pt into a rknn model through codes below: from ultralytics import YOLO model = YOLO("yolo11n.pt") model.export(format="rknn", name="rk3588") and everything went correct. A rknn model named yolo11n-rk3588.rknn was build and used to infer a very traditional yolo demo image "bus"(a blue bus with several people in front of it), but no detection result was obtained.

    I tried to convert the yolol11n.pt into onnx model through above codes firstly, and then used a convert.py offered by rknn_model_zoo/examples/yolo11/python,

    import sys from rknn.api import RKNN

    DATASET_PATH = '../../../datasets/COCO/coco_subset_20.txt' DEFAULT_RKNN_PATH = '/home/yizi/model/yolo11/yolodet.rknn' DEFAULT_QUANT = False

    def parse_arg(): if len(sys.argv) < 3: print("Usage: python3 {} onnx_model_path [platform] [dtype(optional)] [output_rknn_path(optional)]".format(sys.argv[0])) print(" platform choose from [rk3562,rk3566,rk3568,rk3588,rk3576,rk1808,rv1109,rv1126]") print(" dtype choose from [i8, fp] for [rk3562,rk3566,rk3568,rk3588,rk3576]") print(" dtype choose from [u8, fp] for [rk1808,rv1109,rv1126]") exit(1)

    model_path = sys.argv[1]
    platform = sys.argv[2]
    
    do_quant = DEFAULT_QUANT
    if len(sys.argv) > 3:
        model_type = sys.argv[3]
        if model_type not in ['i8', 'u8', 'fp']:
            print("ERROR: Invalid model type: {}".format(model_type))
            exit(1)
        elif model_type in ['i8', 'u8']:
            do_quant = True
        else:
            do_quant = False
    
    if len(sys.argv) > 4:
        output_path = sys.argv[4]
    else:
        output_path = DEFAULT_RKNN_PATH
    
    return model_path, platform, do_quant, output_path

    if name == 'main': model_path, platform, do_quant, output_path = parse_arg()

    # Create RKNN object
    rknn = RKNN(verbose=False)
    
    # Pre-process config
    print('--> Config model')
    rknn.config(mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform=platform)
    print('done')
    
    # Load model
    print('--> Loading model')
    ret = rknn.load_onnx(model=model_path)
    if ret != 0:
        print('Load model failed!')
        exit(ret)
    print('done')
    
    # Build model
    print('--> Building model')
    ret = rknn.build(do_quantization=do_quant, dataset=DATASET_PATH)
    if ret != 0:
        print('Build model failed!')
        exit(ret)
    print('done')
    
    # Export rknn model
    print('--> Export rknn model')
    ret = rknn.export_rknn(output_path)
    if ret != 0:
        print('Export rknn model failed!')
        exit(ret)
    print('done')
    
    # Release
    rknn.release()

    Still everything went correct. A rknn model was created and applied to "bus.jpg", and there was no detection at all. I wonder what's wrong in the whole process?

    1
    1. Glenn JocherUltralytics

      @yizi72 thanks for the detailed report. For RKNN inference issues with YOLO11, please ensure:

      1. You're using matching RKNN Toolkit and runtime versions (update both if needed)
      2. Input images are 640x640 with pixel values normalized 0-255
      3. You load models via YOLO('./yolo11n_rknn_model') as shown in our RKNN deployment docs

      If issues persist, please share your full environment details and validation results from both export methods.

    2. yizi72GitHub

      1、I used rknn-toolkit2-2.3.0 which is the latest version ,bu I still did "pip install rknn-toolkit2 --upgrade" again. 2、The input image is 640*640, uchar, 0-255. 3. I used the following setting to transfer yolo11n.pt to onnx, from ultralytics import YOLO

      加载一个模型,路径为 YOLO 模型的 .pt 文件

      model = YOLO("yolo11n.pt")

      导出模型,设置多种参数

      model.export( format="onnx", imgsz=(640, 640),
      keras=False,
      optimize=True,
      half=False,
      int8=False,
      dynamic=False,
      simplify=True,
      opset=12,
      #workspace=4.0,
      nms=False,
      batch=1
      ) and applied the following codes to verify yolo11n.onnx model:

      from ultralytics import YOLO from PIL import Image

      Load a model

      model = YOLO("/home/yizi/models/yolo11/yolo11n.onnx") # load an official model

      Predict with the model

      results = model("bus.jpg") # predict on an image #results = rknn_model("bus.jpg")

      Access the results

      for result in results: xywh = result.boxes.xywh # center-x, center-y, width, height xywhn = result.boxes.xywhn # normalized xyxy = result.boxes.xyxy # top-left-x, top-left-y, bottom-right-x, bottom-right-y xyxyn = result.boxes.xyxyn # normalized names = [result.names[cls.item()] for cls in result.boxes.cls.int()] # class name of each box confs = result.boxes.conf # confidence score of each box

      The results are good. All objects in bus.jpg were detected. Then I run the following code to transfer onnx to rknn:

      from rknn.api import RKNN import cv2 import numpy as np

      if name == 'main': rknn = RKNN(verbose=True,verbose_file="log.txt") rknn.config( mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform="rk3588", optimization_level=3 )

      rknn.load_onnx(
          model="/home/yizi/models/yolo11/yolo11n.onnx",
          input_size_list=[[1,3,640,640]], 

      )

      rknn.build(
          do_quantization=False 
      )
      
      rknn.export_rknn(
          export_path="/home/yizi/models/yolo11/yolo11n.rknn" 
      )
      
      rknn.release()

      The transferring process went smooth, two problems appeared.

      Firstly,when this rknn model was pushed into a rk3588 tablet, and run with a binary executable file generated by cross-compilation, no result was detected.

      Secondly, I tried to run the following codes:

      from ultralytics import YOLO from PIL import Image

      rknn_model = YOLO("/home/yizi/models/yolo11/yolo11n.rknn", task=detect)

      results = model("bus.jpg")

      An error message, "model='/home/yizi/models/yolo11/yolo11n.rknn' is not a supported model format. Ultralytics supports: ('PyTorch', 'TorchScript', 'ONNX', 'OpenVINO', 'TensorRT', 'CoreML', 'TensorFlow SavedModel', 'TensorFlow GraphDef', 'TensorFlow Lite', 'TensorFlow Edge TPU', 'TensorFlow.js', 'PaddlePaddle', 'MNN', 'NCNN', 'IMX', 'RKNN') See https://docs.ultralytics.com/modes/predict for help. " appeared.

      Last question, how many outputs does yolo11 yield? one or six?

    3. Glenn JocherUltralytics

      @yizi72 thank you for the detailed report. For RKNN deployment issues:

      1. When using Ultralytics RKNN export, load models via the exported directory (not .rknn file), as shown in our RKNN deployment docs:
      model = YOLO('./yolo11n_rknn_model')  # directory path
      1. Ensure your RK3588 runtime uses matching librknnrt.so versions with your RKNN Toolkit 2.3.0 installation

      2. YOLO11 detection models output 3 tensors by default (different from older YOLO versions). For conversion issues, we recommend using our native model.export(format='rknn') rather than manual ONNX conversion.

      Let me know if you still face issues after verifying these points.

    4. yizi72GitHub

      Thanks for replies. I did verify all points you've mentioned above. When running model("bus.jpg"), error messages show:

      File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/ultralytics/engine/model.py", line 182, in call return self.predict(source, stream, **kwargs) File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/ultralytics/engine/model.py", line 543, in predict self.predictor.setup_model(model=self.model, verbose=is_cli) File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/ultralytics/engine/predictor.py", line 378, in setup_model self.model = AutoBackend( File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/home/yizi/anaconda3/envs/rknn/lib/python3.9/site-packages/ultralytics/nn/autobackend.py", line 495, in init raise OSError("RKNN inference is only supported on Rockchip devices.") OSError: RKNN inference is only supported on Rockchip devices.

      It's very strange because I did connect a RK3588 tablet with my virtual machine.

      I still have a quesiton on yolo11n output: yolo11n.pt has one output with shape (1, 84, 8400), and yolo11n-rk3588.rknn has one output with shape (1,84, 8400, 0). Are they correct? If so, can I use the similar codes to analyze rknn output, such as :

      for result in results: xywh = result.boxes.xywh # center-x, center-y, width, height xywhn = result.boxes.xywhn # normalized xyxy = result.boxes.xyxy # top-left-x, top-left-y, bottom-right-x, bottom-right-y xyxyn = result.boxes.xyxyn # normalized names = [result.names[cls.item()] for cls in result.boxes.cls.int()] # class name of each box confs = result.boxes.conf

    5. Glenn JocherUltralytics

      @yizi72 thanks for the details. For the RKNN inference error: ensure your Python script is running directly on the RK3588 hardware (not through a VM) as RKNN models require native Rockchip NPU access. Regarding outputs, YOLO11 detection models produce 3 output tensors by default (see our RKNN deployment docs), but your code should work if using the exported RKNN directory with the included postprocessing. Let me know if you still face issues after verifying native hardware execution.

  3. Yufeng Wu

    Hello, Ultralytics team. I wonder if it's possible to port yolo26n-pose to rknn? I'm not sure if it's difficult to work out on my own? Is there any model architecture problems that prevent porting the pose models to rknn?

  4. montemadjoGitHub

    It appears it does not support rv1106 platform yet

    When I try to export the model using this command

    model.export(format="rknn", name="rv1106", imgsz=320)

    It gives the following error

    E build: Current target_platform(rv1106) not support do_quantization = False!
    E export_rknn: RKNN model does not exist, please load & build model first!

    🤔

    1. Glenn JocherUltralytics(edited)

      rv1106 is an INT8-only target, so it needs a quantized RKNN build. Older Ultralytics releases always built with do_quantization=False, which fails with the message you saw. Current releases support it: upgrade with pip install -U ultralytics and export with quantize=8 plus calibration data:

      model.export(format="rknn", name="rv1106", imgsz=320, quantize=8, data="coco8.yaml")

      See the Rockchip RKNN export guide. If it still fails, share your !yolo checks output and rknn-toolkit2 version here and we can sanity-check the environment.

      1
    2. montemadjoGitHub(edited)

      Here's the quick workaround for those who want to work with yolo26 on RV1106 chip (like Luckfox pico etc). I ran into a couple of issues there, including quantization erasing my confidence results.

      Long story short - first you need to monkey-patch head.Detect.forward function in order to get two tensors - one for bboxes and one for scores. That way you escape all the heavy non-supported tensor functions. Then you export your yolo26 model to onnx format and finally to rknn like this

      import torch
      from ultralytics.nn.modules import head
      
      # 1. First we monkey-patch the forward function
      def patched_forward(self, x):
          if self.end2end:
              h = self.one2one
          else:
              h = self.one2many
      
          box_head = h["box_head"]
          cls_head = h["cls_head"]
      
          if box_head is None or cls_head is None:
              raise RuntimeError("Head moduls are None")
      
          bs = x[0].shape[0]
          boxes = torch.cat([box_head[i](x[i]).view(bs, 4 * self.reg_max, -1) for i in range(self.nl)], dim=-1)
          scores = torch.cat([cls_head[i](x[i]).view(bs, self.nc, -1) for i in range(self.nl)], dim=-1)
      
          preds = {"boxes": boxes, "scores": scores, "feats": x}
          y = self._inference(preds)
      
          # We are splitting into 2 tensors - both gets its own scale/zp at INT8 quantization
          bboxes = y[:, :4, :]   # [1, 4, 2100] — range 0-640
          scores = y[:, 4:, :]   # [1, 1, 2100] — range 0-1 (already sigmoid-ed)
          score_max_idx = scores[0, 0, :].argmax()
          return bboxes, scores
      
      head.Detect.forward = patched_forward
      
      # 2. Then we export to onnx model (we got best.pt model from ultralytics platform)
      from ultralytics import YOLO
      model = YOLO("best.pt")
      model.export(format="onnx", dynamic=False, opset=12, imgsz=320)
      
      # 3. Then we export to RKNN format
      from rknn.api import RKNN
      
      rknn = RKNN()
      
      # 3.1. Konfiguracija specifična za RV1106
      rknn.config(
          mean_values=[[0, 0, 0]], 
          std_values=[[255, 255, 255]], 
          target_platform='rv1106',
          quantized_algorithm='normal', # Ili 'mmse' za bolju preciznost
          quantized_dtype='asymmetric_quantized-8'
      )
      
      # 3.2. Load the ONNX model
      print("Loading ONNX model...")
      rknn.load_onnx(model='best.onnx')
      
      # 3.3. Build modela sa INT8 kvantizacijom
      # calib.txt should contain paths to calibration images
      print("Building RKNN model with quantization...")
      rknn.build(do_quantization=True, dataset='./calib.txt')
      
      # 3.4. Exporting final .rknn file
      print("Exporting RKNN model...")
      rknn.export_rknn('./yolo26_rv1106_model_optimized.rknn')
      
      rknn.release()

      After this you need to postprocess those two outputs like this

      yolo26_postprocess.h

      /**
       * YOLO26 Post-processing header for Fish Counter
       *
       * Supports YOLO26 2-output format: boxes [1, 4, N] + scores [1, nc, N]
       *
       * Based on the working rknn_minimal_test example.
       */
      
      #ifndef _YOLO26_POSTPROCESS_H_
      #define _YOLO26_POSTPROCESS_H_
      
      #include <stdint.h>
      #include <vector>
      #include "rknn_api.h"
      #include "rknn_detector.h"
      
      // YOLO26 model configuration defaults
      #define YOLO26_CONF_THRESHOLD 0.25f
      #define YOLO26_NMS_THRESHOLD 0.45f
      
      // Detection result (same structure as postprocess.h for compatibility)
      typedef struct
      {
          float x1, y1, x2, y2; // Bounding box (in model coordinates)
          float confidence;
          int class_id;
      } yolo26_detection_t;
      
      // YOLO26 context for format detection and caching
      typedef struct
      {
          int num_predictions;
          int num_classes;
          bool boxes_transposed; // true = [1, 4, N], false = [1, N, 4]
      } yolo26_context_t;
      
      /**
       * Initialize YOLO26 post-processing
       * Detects the output format from model attributes
       *
       * @param app_ctx The RKNN app context with model info
       * @param yolo26_ctx Output: YOLO26 context for format info
       * @return 0 on success, -1 on error
       */
      int yolo26_init_postprocess(rknn_app_context_t *app_ctx, yolo26_context_t *yolo26_ctx);
      
      /**
       * Deinitialize YOLO26 post-processing
       */
      void yolo26_deinit_postprocess();
      
      /**
       * YOLO26 post_process: processes model outputs and returns detections
       * Uses 2-output format: boxes [1, 4, N] + scores [1, nc, N]
       *
       * @param app_ctx The RKNN app context
       * @param yolo26_ctx YOLO26 context from init
       * @param outputs Pointer to output tensor memory array
       * @param letter_box Letterbox transform info for coordinate mapping
       * @param conf_threshold Confidence threshold
       * @param nms_threshold NMS threshold
       * @param od_results Output: detection results (compatible with existing code)
       * @return 0 on success, -1 on error
       */
      int yolo26_post_process(rknn_app_context_t *app_ctx, yolo26_context_t *yolo26_ctx,
                              void *outputs, letterbox_t *letter_box,
                              float conf_threshold, float nms_threshold,
                              object_detect_result_list *od_results);
      
      #endif // _YOLO26_POSTPROCESS_H_

      yolo26_postprocess.cc

      /**
       * YOLO26 Post-processing for Fish Counter on RV1106
       *
       * Supports YOLO26 2-output format: boxes [1, 4, N] + scores [1, nc, N]
       *
       * Based on the working rknn_minimal_test example.
       */
      
      #include "yolo26_postprocess.h"
      
      #include <math.h>
      #include <stdint.h>
      #include <stdio.h>
      #include <stdlib.h>
      #include <string.h>
      #include <algorithm>
      #include <vector>
      
      // Clamp value to range
      inline static int clamp(float val, int min, int max)
      {
          return val > min ? (val < max ? val : max) : min;
      }
      
      // Dequantize INT8 to float
      static inline float dequantize(int8_t qnt, int32_t zp, float scale)
      {
          return ((float)qnt - (float)zp) * scale;
      }
      
      // Sigmoid function
      static inline float sigmoid(float x)
      {
          return 1.0f / (1.0f + expf(-x));
      }
      
      // Calculate IoU between two boxes
      static float calculate_iou(float x1_a, float y1_a, float x2_a, float y2_a,
                                 float x1_b, float y1_b, float x2_b, float y2_b)
      {
          float x1 = std::max(x1_a, x1_b);
          float y1 = std::max(y1_a, y1_b);
          float x2 = std::min(x2_a, x2_b);
          float y2 = std::min(y2_a, y2_b);
      
          float inter_w = std::max(0.0f, x2 - x1);
          float inter_h = std::max(0.0f, y2 - y1);
          float inter_area = inter_w * inter_h;
      
          float area_a = (x2_a - x1_a) * (y2_a - y1_a);
          float area_b = (x2_b - x1_b) * (y2_b - y1_b);
          float union_area = area_a + area_b - inter_area;
      
          return union_area > 0 ? inter_area / union_area : 0;
      }
      
      // Non-Maximum Suppression for YOLO26 detections
      static std::vector<yolo26_detection_t> yolo26_nms(std::vector<yolo26_detection_t> &detections, float nms_threshold)
      {
          // Sort by confidence (descending)
          std::sort(detections.begin(), detections.end(),
                    [](const yolo26_detection_t &a, const yolo26_detection_t &b)
                    { return a.confidence > b.confidence; });
      
          std::vector<yolo26_detection_t> result;
          std::vector<bool> suppressed(detections.size(), false);
      
          for (size_t i = 0; i < detections.size(); i++)
          {
              if (suppressed[i])
                  continue;
      
              result.push_back(detections[i]);
      
              for (size_t j = i + 1; j < detections.size(); j++)
              {
                  if (suppressed[j])
                      continue;
      
                  // Only suppress same class
                  if (detections[i].class_id != detections[j].class_id)
                      continue;
      
                  float iou = calculate_iou(
                      detections[i].x1, detections[i].y1, detections[i].x2, detections[i].y2,
                      detections[j].x1, detections[j].y1, detections[j].x2, detections[j].y2);
      
                  if (iou > nms_threshold)
                  {
                      suppressed[j] = true;
                  }
              }
          }
      
          return result;
      }
      
      // Parse YOLO26 2-output format: boxes [1, 4, N] + scores [1, nc, N]
      static std::vector<yolo26_detection_t> parse_yolo26_2output(
          rknn_tensor_mem **output_mems,
          rknn_tensor_attr *output_attrs,
          int model_width, int model_height,
          float conf_threshold, int num_classes,
          bool boxes_transposed)
      {
          std::vector<yolo26_detection_t> detections;
      
          int8_t *boxes_data = (int8_t *)output_mems[0]->virt_addr;
          int32_t boxes_zp = output_attrs[0].zp;
          float boxes_scale = output_attrs[0].scale;
      
          int8_t *scores_data = (int8_t *)output_mems[1]->virt_addr;
          int32_t scores_zp = output_attrs[1].zp;
          float scores_scale = output_attrs[1].scale;
      
          // Determine number of anchors
          int num_anchors;
          if (boxes_transposed)
          {
              num_anchors = output_attrs[0].dims[2];
          }
          else
          {
              num_anchors = output_attrs[0].dims[1];
          }
      
          // Check if scores are already sigmoid or logits
          float scores_min = 9999, scores_max = -9999;
          for (int i = 0; i < (int)output_attrs[1].n_elems; i++)
          {
              float val = dequantize(scores_data[i], scores_zp, scores_scale);
              if (val < scores_min)
                  scores_min = val;
              if (val > scores_max)
                  scores_max = val;
          }
          bool scores_are_sigmoid = (scores_min >= -0.1f && scores_max <= 1.1f);
      
          for (int i = 0; i < num_anchors; i++)
          {
              // Get box coordinates
              float x1, y1, x2, y2;
              if (boxes_transposed)
              {
                  // [1, 4, N] - channel first
                  x1 = dequantize(boxes_data[0 * num_anchors + i], boxes_zp, boxes_scale);
                  y1 = dequantize(boxes_data[1 * num_anchors + i], boxes_zp, boxes_scale);
                  x2 = dequantize(boxes_data[2 * num_anchors + i], boxes_zp, boxes_scale);
                  y2 = dequantize(boxes_data[3 * num_anchors + i], boxes_zp, boxes_scale);
              }
              else
              {
                  // [1, N, 4] - anchor first
                  x1 = dequantize(boxes_data[i * 4 + 0], boxes_zp, boxes_scale);
                  y1 = dequantize(boxes_data[i * 4 + 1], boxes_zp, boxes_scale);
                  x2 = dequantize(boxes_data[i * 4 + 2], boxes_zp, boxes_scale);
                  y2 = dequantize(boxes_data[i * 4 + 3], boxes_zp, boxes_scale);
              }
      
              // Check if format is xywh instead of xyxy
              if (x2 < x1 || y2 < y1)
              {
                  // Assume xywh format: convert to xyxy
                  float cx = x1, cy = y1, w = x2, h = y2;
                  x1 = cx - w / 2.0f;
                  y1 = cy - h / 2.0f;
                  x2 = cx + w / 2.0f;
                  y2 = cy + h / 2.0f;
              }
      
              // Find max class score
              float max_score = -9999;
              int max_class = 0;
              for (int c = 0; c < num_classes; c++)
              {
                  float score;
                  if (boxes_transposed)
                  {
                      // [1, nc, N]
                      score = dequantize(scores_data[c * num_anchors + i], scores_zp, scores_scale);
                  }
                  else
                  {
                      // [1, N, nc]
                      score = dequantize(scores_data[i * num_classes + c], scores_zp, scores_scale);
                  }
      
                  if (!scores_are_sigmoid)
                  {
                      score = sigmoid(score);
                  }
      
                  if (score > max_score)
                  {
                      max_score = score;
                      max_class = c;
                  }
              }
      
              if (max_score < conf_threshold)
                  continue;
      
              // Clamp to model bounds
              x1 = std::max(0.0f, std::min(x1, (float)model_width));
              y1 = std::max(0.0f, std::min(y1, (float)model_height));
              x2 = std::max(0.0f, std::min(x2, (float)model_width));
              y2 = std::max(0.0f, std::min(y2, (float)model_height));
      
              // Skip invalid boxes
              if (x2 <= x1 || y2 <= y1)
                  continue;
      
              yolo26_detection_t det;
              det.x1 = x1;
              det.y1 = y1;
              det.x2 = x2;
              det.y2 = y2;
              det.confidence = max_score;
              det.class_id = max_class;
      
              detections.push_back(det);
          }
      
          return detections;
      }
      
      int yolo26_init_postprocess(rknn_app_context_t *app_ctx, yolo26_context_t *yolo26_ctx)
      {
          if (!app_ctx || !yolo26_ctx)
          {
              printf("[YOLO26] ERROR: Invalid parameters\n");
              return -1;
          }
      
          memset(yolo26_ctx, 0, sizeof(yolo26_context_t));
      
          int n_output = app_ctx->io_num.n_output;
      
          printf("[YOLO26] Initializing 2-output format (%d outputs)...\n", n_output);
      
          // Print output tensor info
          for (int i = 0; i < n_output; i++)
          {
              printf("  Output %d: dims=[%d,%d,%d,%d], size=%d\n", i,
                     app_ctx->output_attrs[i].dims[0],
                     app_ctx->output_attrs[i].dims[1],
                     app_ctx->output_attrs[i].dims[2],
                     app_ctx->output_attrs[i].dims[3],
                     app_ctx->output_attrs[i].size);
          }
      
          if (n_output != 2)
          {
              printf("[YOLO26] ERROR: Expected 2 outputs (boxes + scores), got %d\n", n_output);
              return -1;
          }
      
          // Parse 2-output format: boxes + scores
          int boxes_dim1 = app_ctx->output_attrs[0].dims[1];
          int boxes_dim2 = app_ctx->output_attrs[0].dims[2];
          int scores_dim1 = app_ctx->output_attrs[1].dims[1];
          int scores_dim2 = app_ctx->output_attrs[1].dims[2];
      
          if (boxes_dim1 == 4)
          {
              // [1, 4, N] boxes format
              yolo26_ctx->num_predictions = boxes_dim2;
              yolo26_ctx->boxes_transposed = true;
              yolo26_ctx->num_classes = scores_dim1;
              printf("[YOLO26] Format: boxes [1,4,%d], scores [1,%d,%d]\n",
                     boxes_dim2, scores_dim1, scores_dim2);
          }
          else if (boxes_dim2 == 4)
          {
              // [1, N, 4] boxes format
              yolo26_ctx->num_predictions = boxes_dim1;
              yolo26_ctx->boxes_transposed = false;
              yolo26_ctx->num_classes = scores_dim2;
              printf("[YOLO26] Format: boxes [1,%d,4], scores [1,%d,%d]\n",
                     boxes_dim1, scores_dim1, scores_dim2);
          }
          else
          {
              printf("[YOLO26] ERROR: Invalid boxes dimensions [1,%d,%d], expected [1,4,N] or [1,N,4]\n",
                     boxes_dim1, boxes_dim2);
              return -1;
          }
      
          printf("[YOLO26] Predictions: %d, Classes: %d\n",
                 yolo26_ctx->num_predictions, yolo26_ctx->num_classes);
      
          return 0;
      }
      
      void yolo26_deinit_postprocess()
      {
          // Nothing to cleanup currently
      }
      
      int yolo26_post_process(rknn_app_context_t *app_ctx, yolo26_context_t *yolo26_ctx,
                              void *outputs, letterbox_t *letter_box,
                              float conf_threshold, float nms_threshold,
                              object_detect_result_list *od_results)
      {
          if (!app_ctx || !yolo26_ctx || !outputs || !letter_box || !od_results)
          {
              return -1;
          }
      
          rknn_tensor_mem **output_mems = (rknn_tensor_mem **)outputs;
      
          int model_width = app_ctx->model_width;
          int model_height = app_ctx->model_height;
      
          // Parse 2-output format
          std::vector<yolo26_detection_t> raw_detections = parse_yolo26_2output(
              output_mems,
              app_ctx->output_attrs,
              model_width, model_height,
              conf_threshold,
              yolo26_ctx->num_classes > 0 ? yolo26_ctx->num_classes : 1,
              yolo26_ctx->boxes_transposed);
      
          // Apply NMS
          std::vector<yolo26_detection_t> final_detections = yolo26_nms(raw_detections, nms_threshold);
      
          // Convert to output format with letterbox coordinate mapping
          memset(od_results, 0, sizeof(object_detect_result_list));
          int count = 0;
      
          for (const auto &det : final_detections)
          {
              if (count >= OBJ_NUMB_MAX_SIZE)
                  break;
      
              // Map coordinates back to original image space
              float x1 = det.x1 - letter_box->x_pad;
              float y1 = det.y1 - letter_box->y_pad;
              float x2 = det.x2 - letter_box->x_pad;
              float y2 = det.y2 - letter_box->y_pad;
      
              // Scale back to original coordinates
              od_results->results[count].box.left = (int)(clamp(x1, 0, model_width) / letter_box->scale);
              od_results->results[count].box.top = (int)(clamp(y1, 0, model_height) / letter_box->scale);
              od_results->results[count].box.right = (int)(clamp(x2, 0, model_width) / letter_box->scale);
              od_results->results[count].box.bottom = (int)(clamp(y2, 0, model_height) / letter_box->scale);
              od_results->results[count].prop = det.confidence;
              od_results->results[count].cls_id = det.class_id;
      
              count++;
          }
      
          od_results->count = count;
      
          return 0;
      }

      And you're good to go! 🚀

    3. Glenn JocherUltralytics(edited)

      Thanks for sharing this—splitting the head into 2 outputs (boxes + scores) is a sensible RV1106 workaround to keep RKNN-friendly ops and avoid INT8 scale/zero-point coupling issues; just note that monkey-patching head.Detect.forward is inherently version-fragile. Current Ultralytics already provides an INT8 flow for RV1106 via quantize=8 with data= calibration images (see the Rockchip RKNN export guide), so please try that first. If you still need your custom split, open a GitHub issue linking back to this thread and include !yolo checks, your rknn-toolkit2 version, and a minimal calibration set description so we can evaluate it.

  5. dttvn0010GitHub

    99ms per image is too bad. Using rknn native with 3 npu cores, I can achieve ~11ms per image

    1. Glenn JocherUltralytics

      @Arshapjoy thanks for the detailed info; for YOLO11 detection heads the export output shape is (1, nc + 4, 8400) (4 box coords plus nc class scores), so with 2 classes (1, 6, 8400) is expected and both classes are present, whereas a postprocessor assuming 5 + nc (old YOLO layout with an extra objectness channel) will effectively drop one class. I’d suggest updating your RKNN-side decoder to treat channels [0:4] as [x, y, w, h] and [4:] as class logits for all nc classes; if you still see a true (1, 7, 8400) PyTorch output on the same checkpoint, please share your ultralytics version and yolo checks output so we can dig deeper. The overall YOLO11 head behaviour is also outlined in the YOLO11 model documentation.

      @BakhtiariB the high-level YOLO("./yolo11n_rknn_model") interface is designed to keep preprocessing, NPU inference and postprocessing together, so if you need to run only the core RKNN inference on the NPU while fully controlling pre/post on the CPU you’ll need to load the exported .rknn with Rockchip’s RKNN API directly and implement your own preprocessing and YOLO11 postprocessing around it.

      @dttvn0010 the ~99 ms figure is a conservative end-to-end pipeline measurement on our RK3588 test setup, so seeing ~11 ms from a highly tuned RKNN deployment using all three NPU cores is quite reasonable; if you are open to sharing your exact model variant, quantization mode, input size and NPU core configuration, we can use that to cross-check and better explain the gap between baseline and fully optimized numbers.

      @NikolayKash in practice, large gains like that usually come from combining INT8-quantized RKNN models, all three NPU cores enabled, a suitable input resolution and very lean Python-side pre/postprocessing, so it would be great if @dttvn0010 can share their exact script and RK3588 configuration for others to replicate.

  6. guojieliGitHub

    he YOLO11 benchmark was run by the Ultralytics team on a Radxa Rock 5B based on the Rockchip RK3588, using the RKNN model format to evaluate speed and accuracy. Could you please clarify whether this was an INT8-quantized model?

  7. ArshapjoyGitHub

    Exported RKNN/ONNX model only has 1 output class instead of 2


    Overview

    I’m using Ultralytics YOLOv11s with two custom classes and exporting it to ONNX and RKNN formats. After conversion, inference on the RKNN device only detects a single class, and the output tensor shape is (1, 6, 8400) instead of the expected (1, 7, 8400). I’ve validated that the source .pt model correctly reports 2 classes, but both ONNX and RKNN exports drop one class.


    Model Details

    • Checkpoint path: path/to/yolo11s_custom.pt
    • Number of classes: 2
    • Input size: 640×640

    from ultralytics import YOLO

    Load the YOLO11 model

    model = YOLO("custom_yolo11s.pt")

    Export the model to RKNN format

    rknn_path = model.export(format="rknn", name="rk3588") print(f"RKNN model saved to: {rknn_path}")

    Actual Behavior

    • Received ONNX , RKNN and metadata.yaml after conversion
    • Exported ONNX model output shape: (1, 6, 8400)
    • Exported RKNN model output shape: (1, 6, 8400)
    • Only one class is detected during inference on the RK3588 board.

    7. Logs & Metadata

    • metadata.yaml generated alongside the RKNN model:
      model:
        name: yolo11s_custom
        format: rknn
        input_shape: [1, 3, 640, 640]
        output_shape: [1, 6, 8400]
      classes: 2
    • ONNX graph info: Attached minimal onnx.shape_inference log shows only 6 channels in the final layer.

    8. Additional Notes

    • Manual ONNX export yields the same issue, so it’s unlikely to be an Ultralytics export bug alone.
    • I’ve confirmed that the PyTorch forward pass on the .pt checkpoint produces a shape (1, 7, 8400).

    9. Request Could you please:

    1. Confirm whether there’s an option/flag in the YOLO-Ultralytics or RKNN export pipeline I’m missing to preserve all classes.
    2. Suggest any workarounds or patches to ensure correct output-channel dimensionality.
    3. Identify if this might be a known issue in RKNN-Toolkit 1.5.0 or Ultralytics YOLO v11.

    Thank you for your assistance!

  8. WeiIIzzGitHub

    Hello, when I exported the rknn model, I performed int8 quantization, but I had an error like this: E build: If set do_quantization = True, dataset file need be provided. But it`s value is None! E export_rknn: RKNN model does not exist, please load build model first! My export code is: from ultralytics import YOLO

    model = YOLO("/home/ky/dfxw/runs/detect/train3/weights/best.pt") model.export(format="rknn", name="rk3576", int8=True, data="/home/ky/rknn_yolo_test/data.yaml") The path pointed to by data is the same as the training dataset. How to solve it

    1. WeiIIzzGitHub

      The content of data.yaml is as follows path: /home/ky/dfxw/split_jiaqian # dataset root dir train: images/train val: images/val

      names: 0: 0 1: 1

    2. Glenn JocherUltralytics(edited)

      @WeiIIzz INT8 export for RKNN is supported with quantize=8 (it replaces the deprecated int8=True) together with data pointing to a dataset YAML, which Ultralytics converts into the calibration image list that RKNN needs:

      from ultralytics import YOLO
      
      model = YOLO("/home/ky/dfxw/runs/detect/train3/weights/best.pt")
      model.export(format="rknn", name="rk3576", quantize=8, data="/home/ky/rknn_yolo_test/data.yaml")

      The error you saw comes from an older version that did not pass a calibration dataset to RKNN, so run pip install -U ultralytics and retry. Make sure the YAML points to existing images. Let us know if this resolves your issue!

    3. WeiIIzzGitHub

      @glenn-jocher I followed your steps to modify the code as follows. But I still encountered the same problem.

      from ultralytics import YOLO
      
      model = YOLO("/home/ky/dfxw/runs/detect/train3/weights/best.pt")
      model.export(format="rknn", name="rk3576", int8=True, data="/home/ky/rknn_yolo_test/images")

      Is there only images in the data path?

    4. Glenn JocherUltralytics(edited)

      @WeiIIzz older releases ignored int8=True for RKNN and built the model without quantization. Current Ultralytics supports INT8 RKNN export, so after pip install -U ultralytics you can export with:

      model.export(format="rknn", name="rk3576", quantize=8, data="data.yaml")

      Here data is used for the calibration images. Without quantize, the export builds a floating-point (FP16) model.

  9. VictorAbobaGitHub

    Good day,

    Is RKNN available for obb tasks? I tried to use it directly, but got an error saying that some operations (cos) are not available in RKNN. As a fix, RKNN suggested adding disable_rules = ['reduce_reshape_op_around_concat'] to my configuration. I actually did this inside the YOLO library. Model was exported first to ONNX and then from ONNX to RKNN works and returns the responses in the format (1,6,21504), which seems what I expected. However, if I use this model through YOLO ("path_to_rknn_model", task = 'obb'), will I still get the results in the standard YOLO format? I just don't want to prescribe pre - and post-processing on my own :)

    Thank you!

    1. Glenn JocherUltralytics(edited)

      @VictorAboba thank you for your question! RKNN export now supports all seven Ultralytics tasks, including OBB, so you can export OBB models directly with format="rknn" and load the exported _rknn_model directory with YOLO(...) to get standard YOLO results without writing your own pre- and post-processing. See the Rockchip RKNN guide for details.

    2. SpidrreGitHub

      Hey! Did you notice any significant decrease in quality of your model after conversion? I am also trying to find a way to convert from OBB onnx to RKNN without 'reduce_reshape_op_around_concat'.

    3. VictorAbobaGitHub

      Hey! Did you notice any significant decrease in quality of your model after conversion? I am also trying to find a way to convert from OBB onnx to RKNN without 'reduce_reshape_op_around_concat'.

      Hello, after @glenn-jocher answer, I didn't attempt to convert the OBB model. We have a few other models, and all of them except YOLO were converted to the RKNN format. We will try using this stuff (YOLO without an NPU and others with it) ^_^ After conversion, the OCR model, for example, has almost the same quality, but it doesn't need 'reduce_reshape_op_around_concat' to be used.