Tencent NCNN 导出
NCNN 是 Tencent 开源的神经网络推理框架,适用于手机、嵌入式开发板和台式计算机。使用 format="ncnn" 导出 Ultralytics YOLO26 模型,会创建一个 yolo26n_ncnn_model/ 目录,其中包含网络文件,由 .param 和 .bin 两个文件组成,权重支持 FP32 或 FP16。
概览#
| 属性 | 详情 |
|---|---|
format 参数 | ncnn |
| 输出 | yolo26n_ncnn_model/,包含 model.ncnn.param、model.ncnn.bin、model_ncnn.py 和 metadata.yaml 的目录 |
| 运行平台 | ARM 和 x86-64 CPU,以及配备 Vulkan 驱动程序的 GPU:手机、Raspberry Pi 等单板计算机,以及台式计算机 |
| 运行时 | NCNN:ncnn Python 包,或用于 Android、iOS 和嵌入式应用的 NCNN C++ 库 |
| 精确率 | FP32 或 FP16 权重;不支持 INT8 导出。YOLO26n 的 FP32 模型为 9.8 MB,使用 FP16 权重时为 5.0 MB |
| 导出主机 | Linux x86-64 或 ARM64、macOS,或 Windows x64;pnnx 为这些平台提供了构建版本。导出不需要 GPU |
Ultralytics Platform 可以帮你将训练好的模型导出为 NCNN,因此你不需要本地导出环境。
快速入门#
安装 Ultralytics 包:
pip install ultralytics如果缺少 ncnn 和 pnnx 包,首次导出会安装这两个包;如果缺少 ncnn,首次使用导出模型进行预测时会安装该包。全新的环境不需要其他操作:记录 Restart runtime or rerun command for updates to take effect 后,首次导出会在同一次运行中完成。导出有以下版本要求:
| 软件包 | 所需版本 |
|---|---|
pnnx | 必须恰好为 20260526。导出会安装此版本并替换其他版本 |
ncnn | 任意版本 |
torch | 2.0 或更高:导出不会检查版本;由于较旧版本会导致推理崩溃,Ultralytics 测试会跳过在这些版本上进行的 NCNN 测试 |
其他安装方式请参阅安装指南。然后导出模型,并加载导出的目录以运行推理或验证模型精度。
from ultralytics import YOLO
# 加载 YOLO26 模型
model = YOLO("yolo26n.pt")
# 将模型导出为 NCNN 格式
model.export(format="ncnn") # creates 'yolo26n_ncnn_model/'
# 或者使用 FP16 权重导出,这样文件大小会减半
model.export(format="ncnn", quantize=16) # creates 'yolo26n_ncnn_model/'两种导出都会写入相同的 yolo26n_ncnn_model/ 目录,因此第二次导出会替换第一次导出的内容。
from ultralytics import YOLO
# 加载导出的 NCNN 模型
model = YOLO("yolo26n_ncnn_model")
# 运行推理
results = model("https://ultralytics.com/images/bus.jpg")from ultralytics import YOLO
# 加载导出的 NCNN 模型
model = YOLO("yolo26n_ncnn_model")
# 在 COCO8 数据集上验证准确率
metrics = model.val(data="coco8.yaml")支持的任务#
NCNN 导出支持所有七种 Ultralytics 任务,以及 YOLOv8、YOLO11 和 YOLO26 模型:目标检测、实例分割、语义分割、深度估计、分类、姿态估计 和 OBB。语义分割和深度估计仅适用于 YOLO26,因为只有 YOLO26 系列包含这两种任务头。YOLOv5u、YOLOv9、YOLOv10 和 YOLO12 的目标检测模型也支持导出。RT-DETR、YOLO-World 和文本提示 YOLOE 模型无法转换,详见限制和故障排查。
导出参数#
| 参数 | 类型 | 默认值 | 说明 |
|---|---|---|---|
format | str | 'ncnn' | 导出模型的目标格式,用于定义与各种部署环境的兼容性。 |
imgsz | int 或 tuple | 640 | 模型输入图像尺寸,在导出的模型中固定:正方形图像用整数表示,或用元组 (height, width) 表示。未传入时,导出会使用检查点训练时采用的尺寸。官方 YOLO26 检查点中,目标检测、分割和姿态模型以及在 ADE20K 上预训练的语义分割模型使用 640,OBB 模型以及在 Cityscapes 上预训练的语义分割模型使用 1024,深度模型使用 768,分类模型使用 224。 |
quantize | int 或 str | None | 存储权重的精度:16 (FP16) 可将文件大小减半;32/未设置时使用 FP32。8 会引发错误,而且使用 quantize=8 训练的检查点无法导出为 NCNN,详见限制和故障排查。此选项取代已弃用的 half 标志。 |
nms | bool,可选 | None | NCNN 模型始终返回原始预测结果,以便在外部执行 NMS。无 NMS 检测头(False)和内置 NMS(True)均不可用,详见限制和故障排查。 |
device | str | None | 用于跟踪模型以进行转换的设备:cpu(默认值,在配备 GPU 的机器上也使用此选项)、用于 CUDA GPU 的 0,或用于 Apple 芯片的 mps。 |
在特定格式的导出参数中,NCNN 仅接受以下参数。所有格式通用的参数请参阅导出模式文档。
精确率#
quantize 用于选择权重在 model.ncnn.bin 中的存储方式。以下尺寸适用于在 imgsz=640 下导出的 YOLO26n 目标检测模型:
quantize | 权重 | YOLO26n 大小 |
|---|---|---|
未设置或 32 | FP32 | 9.8 MB |
16 | FP16 | 5.0 MB |
- FP16 权重:
quantize=16会以半精度存储权重,且无需校准数据。输入和输出仍为float32,因此为 FP32 模型编写的推理代码无需修改即可运行该模型。 - 精度:在 COCO val2017 上,使用 x86-64 服务器 CPU 时,YOLO26n 的 FP32 和 FP16 NCNN 模型、ONNX 模型以及 PyTorch 模型(
rect=False)在相同的 640x640 输入下均达到 40.9 mAPval 50-95。 - FP16 计算:无论存储权重的精度如何,NCNN 也可以使用半精度进行计算。在 Apple M4 上,两个导出模型默认都采用了半精度计算:对于一张测试图像,置信度高于 0.25 的锚点中,FP32 导出的原始框坐标与 ONNX 模型最多相差约 2.1 个像素;在 ncnn 软件包中设置
net.opt.use_fp16_storage = False后,整体输出与其相差不超过 0.01。在 x86-64 服务器 CPU 上,对于相同图像,FP32 导出的原始输出与 ONNX 模型的差异小于 0.03,涵盖目标检测、分割、深度估计、分类、姿态和 OBB 模型。使用 FP16 权重时,这些锚点的框坐标在服务器上与 ONNX 相差约 0.3 个像素,在 M4 上相差约 1.6 个像素。在 M4 上,这些差异带来的精度变化小于 0.1:在 COCO128 上,Ultralytics 验证器对 FP32 导出的模型给出 47.7 mAP 50-95,对使用 FP16 权重的模型也给出 47.7;相同的 640x640 输入下,PyTorch 模型(rect=False)同样为 47.7。
输出格式#
导出会写入一个目录:
| 文件 | 内容 |
|---|---|
model.ncnn.param | 文本格式的网络图 |
model.ncnn.bin | 权重 |
model_ncnn.py | 由 PNNX 编写的 Python 脚本,使用 ncnn 软件包在随机输入上运行模型 |
metadata.yaml | 类别名称、任务、图像尺寸、批次大小、步幅和导出参数;Ultralytics 加载模型时会读取这些信息 |
导出还会在目录中留下一个 __pycache__ 文件夹,你可以将其删除。模型接受一个名为 in0、形状为 (3, height, width) 的输入:一张 RGB 图像,通道优先,float32,缩放至 0-1。输入没有批次维度,输出也没有。以下形状适用于以默认导出尺寸导出的官方 YOLO26n 检查点,其中 Cityscapes 检查点 yolo26n-sem.pt 用于语义分割:
| 任务 | 默认 imgsz | 输出 |
|---|---|---|
| 检测 | 640 | out0 (84, 8400) |
| 分割 | 640 | out0 (116, 8400) 和 out1 (32, 160, 160) |
| 语义 | 1024 | out0 (19, 1024, 1024),每个类别和像素对应一个分数 |
| 深度 | 768 | out0 (1, 768, 768),深度图 |
| 分类 | 224 | out0 (1000,),类别概率 |
| 姿态 | 640 | out0 (56, 8400) |
| OBB | 1024 | out0 (20, 21504) |
- 原始预测结果:目标检测、分割、姿态和 OBB 模型返回与默认 ONNX 导出相同的原始预测结果,但不带批次维度:对于每个锚点,先是输入像素中的框中心 x、中心 y、宽度和高度,然后是类别分数,最后是任务值(32 个掩码系数、17 x 3 个关键点值,或以弧度表示的 OBB 角度)。请自行应用 NMS。
- 分割:
out0存放预测结果,out1存放掩码原型。 - 语义分割:输出包含每个类别和像素对应的分数,而非 ONNX 导出的类别图,因此需要在你自己的代码中沿类别轴取 argmax。
- NMS:不支持无 NMS 或嵌入式 NMS 输出,详见限制和故障排除。
YOLO("yolo26n_ncnn_model") 会从 metadata.yaml 读取布局、添加批次维度并应用 NMS,因此只有在你用自己的代码运行模型时才需要了解这些细节。metadata.yaml 的键包括 names、task、imgsz、stride、batch、end2end、args,以及姿态模型的 kpt_shape。
使用 ncnn 软件包运行#
要在 Ultralytics 软件包之外运行模型,请使用 ncnn Python 软件包加载这两个文件。同样的 .param 和 .bin 文件也可以通过 Android、iOS 和嵌入式 Linux 上的 NCNN C++ 库加载。
# 首先导出:yolo export model=yolo26n.pt format=ncnn
import ncnn
import numpy as np
net = ncnn.Net()
net.opt.num_threads = 4 # 在模型加载前设置选项
assert net.load_param("yolo26n_ncnn_model/model.ncnn.param") == 0
assert net.load_model("yolo26n_ncnn_model/model.ncnn.bin") == 0
image = np.zeros((3, 640, 640), dtype=np.float32) # 经过 letterbox 处理的图像:RGB、CHW、0-1
with net.create_extractor() as ex:
ex.input("in0", ncnn.Mat(image).clone())
_, out0 = ex.extract("out0")
predictions = np.array(out0) # shape (84, 8400),需要进行 NMS 的原始预测结果
assert predictions.shape == (84, 8400), "export a detection model at imgsz=640"ncnn.Mat(image) 会直接读取 NumPy 数组的内存而不进行复制,因此请传入 C 连续的 float32 数组,并在提取完成前保持其有效;或者像上面那样使用 .clone() 复制数组。使用 transpose 创建的数组并不连续,而 astype 或在其后进行除法运算都会保留这种布局,因此请使用 np.ascontiguousarray(x, dtype=np.float32) 完成预处理。在使用此模型进行的测试中,非连续数组运行时没有报错,但返回了错误的预测结果;而 float64 数组则导致 Python 崩溃。
部署#
在目标设备上使用 pip install ultralytics 安装软件包,并将 yolo26n_ncnn_model/ 目录复制到该设备。随后,YOLO("yolo26n_ncnn_model") 会使用 ncnn Python 软件包运行模型,Ultralytics 会在首次使用时安装该软件包。device 参数用于选择 NCNN 的运行位置:
device | 模型运行位置 |
|---|---|
未设置或 cpu | 在 CPU 上运行 NCNN |
vulkan:0, vulkan:1, ... | 在指定索引的 Vulkan 设备上运行 NCNN,详见 Vulkan GPU 推理 |
在配备 CUDA GPU 的机器上运行 0 | 在 CPU 上运行 NCNN:NCNN 没有 CUDA 后端 |
Apple 芯片上的 mps | 在 CPU 上运行 NCNN,同时 Ultralytics 在 Apple GPU 上执行预处理和后处理。在 M4 上,这会使第一张图像的处理时间增加约 0.6 秒,并未加快推理速度,因此请勿设置 device |
- 固定输入形状:模型始终以导出时的
imgsz运行,Ultralytics 会从metadata.yaml中读取该值。传递给predict或val的imgsz会被忽略。要更改图像尺寸,请重新导出。 - 批次:模型一次接受一张图像,Ultralytics 会逐张运行批次中的图像,因此任意数量的图像都可以处理。无论传入何种值,
val都会设置batch=1。 - CPU 线程数:NCNN 会自行选择线程数:Apple M4 上为 4 个,即其性能核心的数量。在 Linux 上,Ultralytics 会将线程数限制为最多 8 个,并且不超过进程可使用的 CPU 数量:包括进程的 CPU 亲和性(
taskset、Docker--cpuset-cpus)以及 cgroup v2 CPU 配额(Docker--cpus)。如需使用不同的线程数,请参阅限制和故障排除。
Vulkan GPU 推理#
NCNN 通过 Vulkan 在 GPU 上运行,支持 AMD、Intel、ARM、Qualcomm 和 Apple 等 GPU 厂商,且无需 CUDA。传入 device="vulkan:0" 可使用第一个 Vulkan 设备,传入 vulkan:1 可使用第二个设备:
from ultralytics import YOLO
# 加载导出的 NCNN 模型
model = YOLO("yolo26n_ncnn_model")
# 在第一个 Vulkan 设备上运行推理
results = model("https://ultralytics.com/images/bus.jpg", device="vulkan:0")GPU 需要 Vulkan 驱动,大多数较新的 GPU 驱动都包含此驱动;在 Linux 上,vulkaninfo 会列出已安装的驱动。索引对应 NCNN 检测到的 Vulkan 设备,这些设备不一定是硬件 GPU。依赖结果前,请先列出设备:
import ncnn
for i in range(ncnn.get_gpu_count()):
print(i, ncnn.get_gpu_info(i).device_name())Vulkan 并不总是比 CPU 快。在 Apple M4 上,YOLO26n 在 Vulkan 设备上每张图像耗时约 31 毫秒,在 CPU 上约 12 毫秒。请在你的设备上分别测量。
部署指南#
- Raspberry Pi:了解如何在 Raspberry Pi 开发板上设置 Ultralytics;该指南会在开发板本机导出为 NCNN。
- NVIDIA Jetson:在 Jetson 设备的 ARM CPU 上运行 NCNN,同时也介绍在其 GPU 上运行 TensorRT。
- NCNN 构建说明:为 Android、iOS、Linux、macOS、Windows 和其他目标平台构建 NCNN C++ 库。
- GitHub 上的 NCNN:源代码、Python 软件包和 C++ 示例。
- 交互式跟踪 UI 示例:一个在纯 CPU 设备上运行 NCNN 模型的 Ultralytics 示例。
- 模型部署选项:将 NCNN 与其他导出格式进行比较。
限制和故障排查#
- INT8:
quantize=8在使用quantize=8 (INT8) is not supported; format='ncnn' supports quantize=16 (FP16), 32 or None (FP32).时会失败。你可以使用 FP16 减小文件大小,或选择支持 INT8 导出的格式,例如 LiteRT 或 MNN。 - 使用
quantize=8训练的检查点:使用量化感知训练微调的检查点在使用format='ncnn' cannot export a QAT model: its Q/DQ ranges are only read by the 'onnx' and 'engine' backends时会失败。请改为导出开始微调前的检查点,或将微调后的检查点导出为 ONNX 或 TensorRT。 - 无 NMS 检测头:NCNN 模型始终返回需要进行 NMS 的原始预测结果。使用
nms=False时,导出 YOLO26 或 YOLOv10 模型会记录This export format does not support end2end models, disabling the end2end branch.并使用一对多检测头;导出其他目标检测、分割、姿态和 OBB 模型(例如 YOLO11)时,则会记录This model has no one-to-one head; using one-to-many outputs. - 嵌入式 NMS:
nms=True会记录format=ncnn does not support embedded NMS; exporting native outputs for external NMS.并写入原始输出。 - 固定输入形状和批次大小:
dynamic=True在使用argument 'dynamic' is not supported for format='ncnn'时会失败,而batch不等于 1 时会因argument 'batch' is not supported for format='ncnn'而失败。要更改图像尺寸,请重新导出。 - 无法转换的模型:RT-DETR(
rtdetr-l.pt)、YOLO-Worldv2(yolov8s-worldv2.pt)和文本提示 YOLOE(yoloe-26s-seg.pt)模型的导出会因NCNN failed to load the exported model, it may contain layers NCNN does not support.而失败。修正文本提示 YOLOE 模型的类别后,可使用set_classes()导出;无提示模型yoloe-26s-seg-pf.pt则可直接导出。 - PNNX 版本:导出需要
pnnx20260526,并会替换较新版本,因为使用 PNNX 20260704 转换的模型会在推理时崩溃。 - 已关闭自动安装:使用
YOLO_AUTOINSTALL=false时,如果环境中没有相关软件包,导出会因No module named 'ncnn'而失败。请使用pip install --no-deps ncnn和pip install pnnx==20260526安装这些软件包。 - 线程数:
predict和val没有用于设置 NCNN 线程数的参数,OMP_NUM_THREADS变量也不会改变线程数。在 Linux 上,Ultralytics 最多使用 8 个线程;在核心数更多的机器上,较大的模型使用更多线程可能运行得更快:在 AMD EPYC 9655 的 16 个核心上通过 ncnn 包运行时,YOLO26x 使用 8 个线程时每张图像耗时约 177 毫秒,使用 16 个线程时约 154 毫秒;YOLO26n 使用两种线程数时耗时均约 23 毫秒。若要选择线程数,请通过 ncnn 包运行模型,并在模型加载前设置net.opt.num_threads。模型加载后再更改num_threads会导致推理变慢。 - 模型目录不完整:如果
model.ncnn.bin缺失或不完整(例如复制到设备时中断),Python 会因段错误而停止,且不显示回溯信息。输出的最后几行会指出原因:fopen yolo26n_ncnn_model/model.ncnn.bin failed或ModelBin read weight_data failed。缺少model.ncnn.param的目录会因StopIteration而失败。请重新复制该目录。 - 不带索引的
device="vulkan":会因IndexError: list index out of range而失败。请像device="vulkan:0"那样写入索引。 - Vulkan 设备不是 GPU:在本页面使用的 Linux 服务器上,NCNN 列出的唯一 Vulkan 设备是
llvmpipe,这是一个在 CPU 上运行的软件渲染器。此时,device="vulkan:0"会正常运行,但并未使用 GPU。使用不存在的索引(例如在只有一个设备的机器上使用vulkan:9)也不会报错。 - 输出目录:每次导出同一个检查点时都会写入相同的
yolo26n_ncnn_model/目录,无论使用的是quantize还是imgsz,因此后一次导出会覆盖前一次导出。若要保留两者,请重命名目录,并在名称中保留_ncnn_model,例如yolo26n_fp16_ncnn_model/,因为 Ultralytics 通过该标识识别格式。缺少该标识的目录,或.param文件的路径,会因is not a supported model format而失败。
其他错误请参阅常见问题指南。
基准测试#
Ultralytics 发布了两个 ARM 平台的 NCNN 基准测试。Raspberry Pi 5 基准测试比较了 YOLO26n 和 YOLO26s 的 NCNN 与其他七种格式;NVIDIA Jetson 基准测试则包含全部五种 YOLO26 目标检测模型的 NCNN 测试。YOLO26 模型页面列出了 PyTorch 检查点的精度。
以下为导出的 YOLO26 目标检测模型大小,以 model.ncnn.param 和 model.ncnn.bin 在 imgsz=640 下的总和表示,单位为 MB(10^6 字节)。导出日志会打印整个目录的大小,YOLO26n 为 9.4 MB,YOLO26n 为 4.8 MB:
| 模型 | PyTorch | NCNN FP32 | NCNN FP16 |
|---|---|---|---|
| YOLO26n | 5.5 | 9.8 | 5.0 |
| YOLO26s | 20.4 | 38.1 | 19.2 |
| YOLO26m | 44.3 | 81.8 | 41.0 |
| YOLO26l | 53.2 | 99.4 | 49.8 |
| YOLO26x | 118.7 | 223.0 | 111.7 |
使用 ultralytics 8.4.215、NCNN 1.0.20261020 和 PNNX 20260526 导出,日期为 2026 年 11 月 16 日。
关于 NCNN#
NCNN 由 Tencent 开发,并应用于 QQ 和 WeChat 等 Tencent 应用中。NCNN 没有第三方运行时依赖,可在 CPU 上运行,也可通过 Vulkan 在 GPU 上运行;它使用 PNNX 工具转换 PyTorch 模型,Ultralytics 导出时也会使用该工具。完整参考资料请参阅 NCNN 文档。
常见问题#
如果你关注模型大小,请使用 FP16(
quantize=16):YOLO26n 使用 FP16 权重时为 5.0 MB,而 FP32 时为 9.8 MB,且无需校准数据。在 COCO val2017 上,YOLO26n 使用 FP16 权重和 FP32 时均达到相同的 40.9 mAPval 50-95。FP16 权重会缩小文件,但不会加快模型运行:NCNN 会根据硬件选择计算精度,详见精度。部署前,请在你自己的数据和设备上验证导出的模型。pnnx和ncnn软件包提供 64 位 ARM Linux 构建版本,Raspberry Pi 指南介绍了如何在开发板上使用相同的format="ncnn"命令进行导出。你也可以在 PC 上导出,然后将yolo26n_ncnn_model/目录复制到开发板上。在 32 位 ARM 系统上,pnnx没有可用的构建版本,因此请在其他机器上导出;ncnn有适用于该系统的构建版本,但 Ultralytics 尚未在该系统上测试。按照 NCNN 构建说明为你的平台构建或下载 NCNN 库,将
model.ncnn.param和model.ncnn.bin添加到应用中,并按照输出格式中的说明输入in0。你的代码需要对图像进行 letterbox 处理、解码原始预测结果并应用 NMS。如果你更希望使用现成的应用组件,Ultralytics YOLO Flutter 插件可以在 Android 上运行 LiteRT 导出模型,并在 iOS 上运行 Core ML 导出模型。
评论37
提出问题、分享结果,或帮助其他人。
gpsamt(已编辑)
I'm testing the use of ncnn on android. I converted a yolov11 model. I've tried every way to adjust the code, but I can't. The detections are not coming out correctly. The ncnn model on windows works as can be seen in the image. I'm sharing my native code in c++. If anyone can help me I'd really appreciate it.
Image detected with yolo predict model=C:\Users\guips\Downloads\best_ncnn_model source="C:\Users\guips\Downloads\brasil\test822.jpg" save_txt=true
#include <jni.h> #include <android/log.h> #include #include #include #include <opencv2/opencv.hpp> #include <ncnn/net.h> // #include <stb_image.h> // Uncomment if using stb_image
#define LOG_TAG "NCNN_DETECTOR" #define LOGD(...) __android_log_print(ANDROID_LOG_DEBUG, LOG_TAG, VA_ARGS) #define LOGI(...) __android_log_print(ANDROID_LOG_INFO, LOG_TAG, VA_ARGS) #define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, LOG_TAG, VA_ARGS)
static ncnn::Net g_net; static std::vectorstd::string g_labels; static bool g_model_loaded = false;
struct Detection { float x, y, width, height; int label; float prob; };
static float sigmoid(float x) { return 1.0f / (1.0f + std::exp(-x)); }
static float calculate_iou(const Detection& a, const Detection& b) { float x1 = std::max(a.x, b.x); float y1 = std::max(a.y, b.y); float x2 = std::min(a.x + a.width, b.x + b.width); float y2 = std::min(a.y + a.height, b.y + b.height);
float inter_area = std::max(0.0f, x2 - x1) * std::max(0.0f, y2 - y1); float union_area = a.width * a.height + b.width * b.height - inter_area; return union_area > 0 ? inter_area / union_area : 0.0f;}
extern "C" JNIEXPORT jboolean JNICALL Java_com_example_ncnnmy_1ok_MainActivity_loadModel( JNIEnv* env, jobject thiz, jstring param_path, jstring bin_path, jstring labels_path) {
const char* param_path_str = env->GetStringUTFChars(param_path, nullptr); const char* bin_path_str = env->GetStringUTFChars(bin_path, nullptr); const char* labels_path_str = env->GetStringUTFChars(labels_path, nullptr); LOGI("Starting model loading..."); LOGD("Param path: %s", param_path_str); LOGD("Bin path: %s", bin_path_str); LOGD("Labels path: %s", labels_path_str); g_net.clear(); g_labels.clear(); g_model_loaded = false; g_net.opt.num_threads = 4; g_net.opt.use_vulkan_compute = false; g_net.opt.use_fp16_packed = false; g_net.opt.use_fp16_storage = false; g_net.opt.use_fp16_arithmetic = false; int ret = g_net.load_param(param_path_str); if (ret != 0) { LOGE("Failed to load .param file (%d)", ret); env->ReleaseStringUTFChars(param_path, param_path_str); env->ReleaseStringUTFChars(bin_path, bin_path_str); env->ReleaseStringUTFChars(labels_path, labels_path_str); return JNI_FALSE; } ret = g_net.load_model(bin_path_str); if (ret != 0) { LOGE("Failed to load .bin file (%d)", ret); env->ReleaseStringUTFChars(param_path, param_path_str); env->ReleaseStringUTFChars(bin_path, bin_path_str); env->ReleaseStringUTFChars(labels_path, labels_path_str); return JNI_FALSE; } FILE* fp = fopen(labels_path_str, "r"); if (!fp) { LOGE("Failed to open labels file: %s", labels_path_str); env->ReleaseStringUTFChars(param_path, param_path_str); env->ReleaseStringUTFChars(bin_path, bin_path_str); env->ReleaseStringUTFChars(labels_path, labels_path_str); return JNI_FALSE; } char buffer[256]; while (fgets(buffer, sizeof(buffer), fp) != nullptr) { std::string label(buffer); label.erase(std::remove(label.begin(), label.end(), '\n'), label.end()); label.erase(std::remove(label.begin(), label.end(), '\r'), label.end()); if (!label.empty()) { g_labels.push_back(label); } } fclose(fp); if (g_labels.empty()) { LOGE("No labels loaded from %s", labels_path_str); env->ReleaseStringUTFChars(param_path, param_path_str); env->ReleaseStringUTFChars(bin_path, bin_path_str); env->ReleaseStringUTFChars(labels_path, labels_path_str); return JNI_FALSE; } LOGI("Model loaded successfully"); LOGI("Total labels loaded: %zu", g_labels.size()); g_model_loaded = true; env->ReleaseStringUTFChars(param_path, param_path_str); env->ReleaseStringUTFChars(bin_path, bin_path_str); env->ReleaseStringUTFChars(labels_path, labels_path_str); return JNI_TRUE;}
extern "C" JNIEXPORT jstring JNICALL Java_com_example_ncnnmy_1ok_MainActivity_detect( JNIEnv* env, jobject thiz, jstring image_path, jdouble conf_threshold, jdouble nms_threshold) {
LOGI("Running updated detect method (no debug image)"); if (!g_model_loaded) { LOGE("Model not loaded"); return env->NewStringUTF("error:model_not_loaded"); } const char* image_path_str = env->GetStringUTFChars(image_path, nullptr); LOGI("Processing image: %s", image_path_str); // Option 1: Use OpenCV (current) cv::Mat image = cv::imread(image_path_str, cv::IMREAD_COLOR); env->ReleaseStringUTFChars(image_path, image_path_str); /* // Option 2: Use stb_image (uncomment to test) int width, height, channels; unsigned char* image_data = stbi_load(image_path_str, &width, &height, &channels, 3); env->ReleaseStringUTFChars(image_path, image_path_str); if (!image_data) { LOGE("Failed to load image with stb_image"); return env->NewStringUTF("error:image_load_failed"); } cv::Mat image(height, width, CV_8UC3, image_data); */ if (image.empty()) { LOGE("Failed to load image"); // stbi_image_free(image_data); // Uncomment if using stb_image return env->NewStringUTF("error:image_load_failed"); } LOGD("Input image: %dx%d, channels=%d", image.cols, image.rows, image.channels()); const int target_size = 640; int img_w = image.cols; int img_h = image.rows; float scale = std::min((float)target_size / img_w, (float)target_size / img_h); int new_w = static_cast<int>(img_w * scale); int new_h = static_cast<int>(img_h * scale); int dx = (target_size - new_w) / 2; int dy = (target_size - new_h) / 2; cv::Mat resized; cv::resize(image, resized, cv::Size(new_w, new_h)); cv::Mat input_mat(target_size, target_size, CV_8UC3, cv::Scalar(114, 114, 114)); resized.copyTo(input_mat(cv::Rect(dx, dy, new_w, new_h))); cv::cvtColor(input_mat, input_mat, cv::COLOR_BGR2RGB); input_mat.convertTo(input_mat, CV_32FC3, 1.0 / 255.0); ncnn::Mat in = ncnn::Mat::from_pixels(input_mat.data, ncnn::Mat::PIXEL_RGB, target_size, target_size); // stbi_image_free(image_data); // Uncomment if using stb_image ncnn::Extractor ex = g_net.create_extractor(); ex.input("in0", in); ncnn::Mat out; ex.extract("out0", out); if (out.empty()) { LOGE("Empty output from model"); return env->NewStringUTF("error:empty_model_output"); } LOGD("Output dimensions: w=%d, h=%d, c=%d", out.w, out.h, out.c); int num_preds, num_classes; bool is_decoupled = false; if (out.c == 1) { LOGD("Possible decoupled heads detected, checking for additional outputs..."); ncnn::Mat boxes, scores, classes; ex.extract("boxes", boxes); ex.extract("scores", scores); ex.extract("classes", classes); if (!boxes.empty() && !scores.empty() && !classes.empty()) { is_decoupled = true; LOGD("Decoupled heads detected: boxes=%dx%d, scores=%dx%d, classes=%dx%d", boxes.w, boxes.c, scores.w, scores.c, classes.w, classes.c); return env->NewStringUTF("error:decoupled_heads_not_implemented"); } } if (!is_decoupled) { if (out.c >= 5) { num_preds = out.h * out.w; num_classes = out.c - 5; LOGD("Interpreting as [1, num_predictions, 5 + num_classes]: num_preds=%d, num_classes=%d", num_preds, num_classes); } else if (out.h >= 5) { num_preds = out.w; num_classes = out.h - 5; LOGD("Interpreting as [1, 5 + num_classes, num_predictions]: num_preds=%d, num_classes=%d", num_preds, num_classes); } else { LOGE("Invalid output dimensions: w=%d, h=%d, c=%d", out.w, out.h, out.c); return env->NewStringUTF("error:invalid_output_dimensions"); } } if (num_classes != g_labels.size()) { LOGE("Mismatch between model outputs (%d) and labels (%zu)", num_classes, g_labels.size()); return env->NewStringUTF("error:model_labels_mismatch"); } std::vector<Detection> detections; int stride = (out.c >= 5) ? out.c : out.h; const float* data = (float*)out.data; // Log raw outputs if (num_preds > 0) { LOGD("Raw output sample: x=%.3f, y=%.3f, w=%.3f, h=%.3f, obj=%.3f", data[0], data[1], data[2], data[3], data[4]); // Log first class score if (num_classes > 0) { LOGD("Raw class score sample: class_0=%.3f", data[5]); } } for (int i = 0; i < num_preds; i++) { const float* pred = data + i * stride; float obj_score = sigmoid(pred[4]); if (obj_score < conf_threshold) continue; int class_id = 0; float max_score = sigmoid(pred[5]); for (int j = 1; j < num_classes; j++) { float score = sigmoid(pred[5 + j]); if (score > max_score) { max_score = score; class_id = j; } } float confidence = obj_score * max_score; if (confidence < conf_threshold) continue; // Normalize coordinates with sigmoid (assuming logits) float x_center = sigmoid(pred[0]) * target_size; float y_center = sigmoid(pred[1]) * target_size; float width = sigmoid(pred[2]) * target_size; float height = sigmoid(pred[3]) * target_size; // Adjust for letterbox float x = (x_center - width / 2 - dx) / scale; float y = (y_center - height / 2 - dy) / scale; width /= scale; height /= scale; LOGD("Prediction %d: x_center=%.2f, y_center=%.2f, width=%.2f, height=%.2f", i, x_center, y_center, width, height); LOGD("Transformed %d: x=%.2f, y=%.2f, width=%.2f, height=%.2f", i, x, y, width, height); // Validate coordinates if (x < 0 || y < 0 || width <= 0 || height <= 0 || x + width > img_w || y + height > img_h) { LOGD("Skipping invalid detection %d: x=%.2f, y=%.2f, w=%.2f, h=%.2f", i, x, y, width, height); continue; } // Minimum size filter if (width < 50 || height < 50) { LOGD("Skipping small detection %d: w=%.2f, h=%.2f", i, width, height); continue; } detections.push_back({x, y, width, height, class_id, confidence}); } // NMS std::sort(detections.begin(), detections.end(), [](const Detection& a, const Detection& b) { return a.prob > b.prob; }); std::vector<bool> suppressed(detections.size(), false); std::vector<Detection> final_detections; for (size_t i = 0; i < detections.size(); i++) { if (suppressed[i]) continue; final_detections.push_back(detections[i]); for (size_t j = i + 1; j < detections.size(); j++) { if (suppressed[j]) continue; float iou = calculate_iou(detections[i], detections[j]); if (iou > nms_threshold) { suppressed[j] = true; } } } LOGD("Detections after NMS: %zu", final_detections.size()); // Format results std::string result_str; for (const auto& det : final_detections) { if (!result_str.empty()) result_str += ";"; char buf[256]; snprintf(buf, sizeof(buf), "%s:%.2f,%.0f,%.0f,%.0f,%.0f", g_labels[det.label].c_str(), det.prob, det.x, det.y, det.width, det.height); result_str += buf; } return env->NewStringUTF(result_str.empty() ? "none" : result_str.c_str());}
Here is a log of the output of the flutter application.
[ ] D/NCNN_DETECTOR( 9180): Prediction 3730: x_center=640.00, y_center=640.00, width=640.00, height=640.00 [ ] D/NCNN_DETECTOR( 9180): Transformed 3730: x=320.00, y=240.00, width=640.00, height=640.00 [ ] D/NCNN_DETECTOR( 9180): Skipping invalid detection 3730: x=320.00, y=240.00, w=640.00, h=640.00 [ ] D/NCNN_DETECTOR( 9180): Prediction 3731: x_center=640.00, y_center=640.00, width=640.00, height=640.00 [ ] D/NCNN_DETECTOR( 9180): Transformed 3731: x=320.00, y=240.00, width=640.00, height=640.00 [ ] D/NCNN_DETECTOR( 9180): Skipping invalid detection 3731: x=320.00, y=240.00, w=640.00, h=640.00 [ ] D/NCNN_DETECTOR( 9180): Prediction 3732: x_center=640.00, y_center=640.00, width=640.00, height=640.00 [ ] D/NCNN_DETECTOR( 9180): Transformed 3732: x=320.00, y=240.00, width=640.00, height=640.00 [ ] D/NCNN_DETECTOR( 9180): Skipping invalid detection 3732: x=320.00, y=240.00, w=640.00, h=640.00 [ ] D/NCNN_DETECTOR( 9180): Detections after NMS: 0 [ ] I/flutter ( 9180): Detection result: none [ ] I/flutter ( 9180): STATUS: No objects detected [ ] E/libEGL ( 9180): called unimplemented OpenGL ES API
Glenn JocherUltralytics(已编辑)@gpsamt looking at your logs, the issue appears to be with output format parsing. YOLO11 NCNN outputs have no separate objectness score, and the class scores are already sigmoid-activated, so don't apply sigmoid again. Try parsing each prediction like this:
// Box is x_center, y_center, width, height (no sigmoid needed) float x_center = pred[0]; float y_center = pred[1]; float width = pred[2]; float height = pred[3]; // pred[4 .. 4+nc) are the per-class scores; take the max class as the confidence // Convert from center format to corner format float x = (x_center - width / 2 - dx) / scale; float y = (y_center - height / 2 - dy) / scale;Also verify your model's actual output format by checking if coordinates are normalized (0-1) or absolute (0-640). The repeated 640.00 values suggest coordinate parsing issues. For official Android deployment examples, refer to the NCNN Android documentation.
aram199922
Hey everyone, I have trained yolo11n.pt pretrained model on my custom dataset. Run the detection on a video and got quite good results. Now trying to export ncnn with the given guide, files are being generated correctly, however the detection on the same video seems corrupted. I am trying to detect potholes on road, however ncnn model predict 300 potholes on each frame. Any suggestion?
aram199922
One more note is that I have trained my model with 736p, 640p modes are converting fine. But does this mean I can't conver higher resolution trained model to ncnn?
Glenn JocherUltralytics@aram199922 The issue with your NCNN model detecting 300 potholes likely relates to confidence thresholds. When exporting to NCNN, default thresholds may not transfer properly. Try explicitly setting a higher confidence threshold during inference:
# Load the NCNN model ncnn_model = YOLO("your_model_ncnn_model") # Run inference with higher confidence threshold results = ncnn_model("video.mp4", conf=0.5) # Adjust this value as neededRegarding your 736p model - yes, you can export higher resolution models to NCNN by specifying the custom size during export:
model = YOLO("your_trained_model.pt") model.export(format="ncnn", imgsz=736) # Use your training resolutionThis ensures the exported model maintains the same input dimensions you used during training. Let me know if this helps resolve the issue!
gpsamt
Hello. Does anyone have any example of implementing the converted ncnn model in android?
PabloBR99
Do this work with YOLOv12?
Glenn JocherUltralytics(已编辑)@PabloBR99 YOLO12 isn't recommended for NCNN export due to training instability and performance limitations. For NCNN deployments, we strongly recommend using YOLO26 (or YOLO11), which offer optimized performance and stability across all platforms. You can find the NCNN export guide in our documentation.
gpsamt
I am exporting a YOLOv11s model to ncnn. My model has 4 classes, however, when I perform the export, it appears that it is identifying 3 classes: PyTorch: starting from '/content/drive/MyDrive/IAGeneration/yolo11/runs/detect/train3/weights/best.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 8, 8400) (18.3 MB). In Android Studio, this message appears:
2025-03-04 17:08:58.490 3853-3853 NcnnDetector com.example.agroinsect D Starting detection 2025-03-04 17:08:58.502 3853-3853 NcnnDetector com.example.test D Param file size: 22768 bytes 2025-03-04 17:08:58.504 3853-3853 NcnnDetector com.example.test D Load param result: 0 2025-03-04 17:08:58.504 3853-3853 NcnnDetector com.example.test D Bin file size: 10633344 bytes 2025-03-04 17:08:58.596 3853-3853 NcnnDetector com.example.test D Load model result: 10633344 2025-03-04 17:08:58.597 3853-3853 NcnnDetector com.example.test D Failed to load model file
[ +82 ms] D/CompatChangeReporter( 3853): Compat change id reported: 312399441; UID 10209; state: ENABLED [ +11 ms] D/nativeloader( 3853): Load /data/app/~~yVwXpMN1F9boaEz-oh0Bpw==/com.example.test-J2_0Fz9v3HSf0USV8uxz6Q==/lib/x86_64/libncnn_detector.so using ns clns-7 from class loader (caller=/data/app/~~yVwXpMN1F9boaEz-oh0Bpw==/com.example.test-J2_0Fz9v3HSf0USV8uxz6Q==/base.apk!classes4.dex): ok [ ] D/NcnnDetector( 3853): Library ncnn_detector loaded successfully [ ] D/NcnnDetector( 3853): Starting detection [ +10 ms] D/NcnnDetector( 3853): Param file size: 22768 bytes [ +2 ms] D/NcnnDetector( 3853): Load param result: 0 [ ] D/NcnnDetector( 3853): Bin file size: 10633344 bytes [ +97 ms] D/NcnnDetector( 3853): Load model result: 10633344 [ +1 ms] D/NcnnDetector( 3853): Failed to load model file [ ] I/flutter ( 3853): Native detect result: null [ ] I/flutter ( 3853): Detections received: [] [ ] I/flutter ( 3853): No detections found.
I have repeated the process several times, but I have not been able to resolve it. I found this answer: 'The current problem (Load model result: 37825744) suggests difficulties in exporting or compatibility.'"
Glenn JocherUltralytics@gpsamt thank you for reporting this issue. Let's address both aspects systematically:
-
Class Count Mismatch:
The output shape (1,8,8400) suggests your model is correctly configured for 4 classes (4 classes + 4 box coordinates = 8). This indicates the export itself is successful. The Android implementation might need adjustment to handle the 4-class output format. -
Model Loading Failure:
The "Failed to load model file" error typically indicates either:
- Missing/corrupted
.paramor.binfiles in deployment - Architecture mismatch between PyTorch and NCNN versions
- Incorrect Android NCNN implementation
We recommend:
- Verify successful local inference with your NCNN model using
yolo predict model='yolo11n_ncnn_model'before Android deployment - Ensure you're using the latest NCNN Android library from Tencent's NCNN GitHub repo
- Double-check that all NCNN model files are properly bundled in your Android assets
For specific Android implementation guidance, please consult NCNN's official Android deployment documentation.
-
gpsamt(已编辑)
I made several attempts and ended up not proceeding, however I really wanted to test my trained model running on android with ncnn. I made an export on windows of a yolo11 model trained in colab. To test the converted model yolo --> ncnn I did a test with an image with two objects. C:\Users\guips\Downloads>yolo predict model=C:\Users\guips\Downloads\best_ncnn_model source="C:\Users\guips\Downloads\brasil\test129.jpg" save_txt=true WARNING ⚠️ Unable to automatically guess model task, assuming 'task=detect'. Explicitly define task for your model, i.e. 'task=detect', 'segment', 'classify','pose' or 'obb'. Ultralytics 8.3.105 🚀 Python-3.11.9 torch-2.6.0+cpu CPU (12th Gen Intel Core(TM) i7-12700KF) Loading C:\Users\guips\Downloads\best_ncnn_model for NCNN inference...
image 1/1 C:\Users\guips\Downloads\brasil\test129.jpg: 640x640 2 Car, 103.0ms Speed: 20.4ms preprocess, 103.0ms inference, 24.2ms postprocess per image at shape (1, 3, 640, 640) Results saved to runs\detect\predict6 1 label saved to runs\detect\predict6\labels 💡 Learn more at https://docs.ultralytics.com/modes/predict
When I try to run the same image in the android application with flutter I can't. In TFLITE the output was (1,8,8400) Now it seems to be outputting (1,9,8400). I'm not able to work with the output data. I/flutter (14031): Labels loaded: 4E/libEGL (14031): called unimplemented OpenGL ES API I/flutter (14031): Storage permission denied D/InsetsController(14031): hide(ime(), fromIme=false) I/ImeTracker(14031): com.example.ncnnmy_ok:e29c5e1e: onCancelled at PHASE_CLIENT_ALREADY_HIDDEN E/libEGL (14031): called unimplemented OpenGL ES API D/InsetsController(14031): hide(ime(), fromIme=false) I/ImeTracker(14031): com.example.ncnnmy_ok:6f2d251e: onCancelled at PHASE_CLIENT_ALREADY_HIDDEN E/libEGL (14031): called unimplemented OpenGL ES API D/CompatChangeReporter(14031): Compat change id reported: 312399441; UID 10223; state: ENABLED I/flutter (14031): Image dimensions: 680.0x453.0 I/flutter (14031): Running inference on /data/user/0/com.example.ncnnmy_ok/cache/a6b569b8-5ff1-4466-ad32-49121fb24aa4/38.jpg D/MainActivity(14031): Received method call: detect I/NCNN (14031): Detecting on image: /data/user/0/com.example.ncnnmy_ok/cache/a6b569b8-5ff1-4466-ad32-49121fb24aa4/38.jpg I/NCNN (14031): Original image size: 680x453 I/NCNN (14031): Output dimensions: w=8400, h=9, c=1 I/NCNN (14031): Output saved to /data/user/0/com.example.ncnnmy_ok/cache/a6b569b8-5ff1-4466-ad32-49121fb24aa4/38.jpg_ncnn_output.txt W/NCNN (14031): Output has 4424 channels, expected 8. Using first 8 channels W/NCNN (14031): Invalid box at idx=9: w=0.000000, h=0.000000 W/NCNN (14031): Invalid box at idx=15: w=-0.010868, h=-0.063400 W/NCNN (14031): Invalid box at idx=16: w=-0.630245, h=-0.447460 W/NCNN (14031): Invalid box at idx=17: w=-0.685141, h=-0.837424 . . . W/NCNN (14031): Invalid box at idx=347: w=-0.077838, h=0.846499 F/libc (14031): Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x71c0c2499110 in tid 14031 (ample.ncnnmy_ok), pid 14031 (ample.ncnnmy_ok)
Build fingerprint: 'google/sdk_gphone64_x86_64/emu64xa:16/BP22.250221.010/13193326:user/release-keys' Revision: '0' ABI: 'x86_64' Timestamp: 2025-04-15 01:29:54.291432600+0000 Process uptime: 156s Cmdline: com.example.ncnnmy_ok pid: 14031, tid: 14031, name: ample.ncnnmy_ok >>> com.example.ncnnmy_ok <<< uid: 10223 signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x000071c0c2499110 rax 000071c0c2499100 rbx 000071c247d33be0 rcx 0000000000000004 rdx 0000000000000008 r8 00007ffc6ffc4080 r9 00007ffc6ffc40a0 r10 0000000000e8cada r11 0000000000000213 r12 00007ffc6ff903b0 r13 000071c247d33c88 r14 00007ffc6ff90718 r15 000071c247d33be0 rdi 00007ffc6ff8f840 rsi 0000000000000167 rbp 00007ffc6ff8f970 rsp 00007ffc6ff8f8b0 rip 000071c053d969ea 90 total frames backtrace: #00 pc 00000000000109ea /data/app/~~ygC8oOQ2RMgj_IeB2wDRKw==/com.example.ncnnmy_ok-1Q_3JHdbdZSe1RwVvc5qjQ==/lib/x86_64/libnative-lib.so (BuildId: 63a90d5f64ab095e3ac958c5db79324dfa15e937) #01 pc 000000000000fd3a /data/app/~~ygC8oOQ2RMgj_IeB2wDRKw==/com.example.ncnnmy_ok-1Q_3JHdbdZSe1RwVvc5qjQ==/lib/x86_64/libnative-lib.so (BuildId: 63a90d5f64ab095e3ac958c5db79324dfa15e937) #02 pc 000000000000e8aa /data/app/~~ygC8oOQ2RMgj_IeB2wDRKw==/com.example.ncnnmy_ok-1Q_3JHdbdZSe1RwVvc5qjQ==/lib/x86_64/libnative-lib.so (Java_com_example_ncnnmy_1ok_MainActivity_detect+2634) (BuildId: 63a90d5f64ab095e3ac958c5db79324dfa15e937) #03 pc 000000000022c80b /apex/com.android.art/lib64/libart.so (art_quick_generic_jni_trampoline+219) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #04 pc 0000000000211dd4 /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+756) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #05 pc 0000000000461715 /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+181) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #06 pc 00000000005f74b2 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2402) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #07 pc 0000000000233611 /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+10625) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #08 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #09 pc 00000000000006a4 anonymous:71c395b1d000 (com.example.ncnnmy_ok.MainActivity.configureFlutterEngine$lambda$0+0) #10 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #11 pc 00000000005f6706 /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*)+102) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #12 pc 00000000005f7498 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2376) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #13 pc 0000000000234b05 /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+15989) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #14 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #15 pc 0000000000000604 anonymous:71c395b1d000 (com.example.ncnnmy_ok.MainActivity.$r8$lambda$8rE-0sywqH2S4u1lhbBM5kfBEYE+0) #16 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #17 pc 00000000005f6706 /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*)+102) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #18 pc 00000000005f7498 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2376) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #19 pc 0000000000234b05 /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+15989) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #20 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #21 pc 00000000000005b8 anonymous:71c395b1d000 (com.example.ncnnmy_ok.MainActivity$$ExternalSyntheticLambda0.onMethodCall+0) #22 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #23 pc 00000000005f6706 /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*)+102) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #24 pc 00000000005f7498 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2376) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #25 pc 00000000002346e5 /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+14933) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #26 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #27 pc 00000000001bfabc anonymous:71c054260000 (io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage+0) #28 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #29 pc 00000000005f6706 /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*)+102) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #30 pc 00000000005f7498 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2376) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #31 pc 00000000002346e5 /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+14933) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #32 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #33 pc 00000000001b1954 anonymous:71c054260000 (io.flutter.embedding.engine.dart.DartMessenger.invokeHandler+0) #34 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #35 pc 00000000005f6706 /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*)+102) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #36 pc 00000000005f7498 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2376) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #37 pc 0000000000233611 /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+10625) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #38 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #39 pc 00000000001b19d8 anonymous:71c054260000 (io.flutter.embedding.engine.dart.DartMessenger.lambda$dispatchMessageToQueue$0$io-flutter-embedding-engine-dart-DartMessenger+0) #40 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #41 pc 00000000005f6706 /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*)+102) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #42 pc 00000000005f827f /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2319) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #43 pc 000000000023482d /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+15261) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #44 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #45 pc 00000000001b1168 anonymous:71c054260000 (io.flutter.embedding.engine.dart.DartMessenger$$ExternalSyntheticLambda0.run+0) #46 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #47 pc 00000000009978fb /apex/com.android.art/lib64/libart.so (artQuickToInterpreterBridge+907) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #48 pc 000000000022c99c /apex/com.android.art/lib64/libart.so (art_quick_to_interpreter_bridge+140) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #49 pc 0000000000160efc [anon_shmem:dalvik-jit-code-cache] (offset 0x2000000) (android.os.Handler.handleCallback+156) #50 pc 00000000000718af [anon_shmem:dalvik-jit-code-cache] (offset 0x2000000) (android.os.Handler.dispatchMessage+159) #51 pc 000000000006e9a4 [anon_shmem:dalvik-jit-code-cache] (offset 0x2000000) (android.os.Looper.loopOnce+1348) #52 pc 0000000000212136 /apex/com.android.art/lib64/libart.so (art_quick_invoke_static_stub+806) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #53 pc 000000000046175c /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+252) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #54 pc 00000000005f74b2 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2402) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #55 pc 0000000000234b05 /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+15989) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #56 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #57 pc 000000000027c140 /system/framework/framework.jar (offset 0x1274000) (android.os.Looper.loop+0) #58 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #59 pc 00000000005f6706 /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*)+102) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #60 pc 00000000005f7498 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2376) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #61 pc 0000000000234b05 /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+15989) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #62 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #63 pc 00000000001f1254 /system/framework/framework.jar (offset 0x7000) (android.app.ActivityThread.main+0) #64 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #65 pc 00000000009978fb /apex/com.android.art/lib64/libart.so (artQuickToInterpreterBridge+907) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #66 pc 000000000022c99c /apex/com.android.art/lib64/libart.so (art_quick_to_interpreter_bridge+140) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #67 pc 0000000000212136 /apex/com.android.art/lib64/libart.so (art_quick_invoke_static_stub+806) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #68 pc 000000000046175c /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+252) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #69 pc 000000000088b130 /apex/com.android.art/lib64/libart.so (_jobject* art::InvokeMethod<(art::PointerSize)8>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jobject*, _jobject*, unsigned long)+2240) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #70 pc 00000000007e0f1a /apex/com.android.art/lib64/libart.so (art::Method_invoke(_JNIEnv*, _jobject*, _jobject*, _jobjectArray*) (.__uniq.165753521025965369065708152063621506277)+42) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #71 pc 000000000022c80b /apex/com.android.art/lib64/libart.so (art_quick_generic_jni_trampoline+219) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #72 pc 0000000000211dd4 /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+756) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #73 pc 0000000000461715 /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+181) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #74 pc 00000000005f74b2 /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, bool, art::JValue*)+2402) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #75 pc 0000000000233611 /apex/com.android.art/lib64/libart.so (void art::interpreter::ExecuteSwitchImplCpp(art::interpreter::SwitchImplContext*)+10625) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #76 pc 000000000022eb25 /apex/com.android.art/lib64/libart.so (ExecuteSwitchImplAsm+5) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #77 pc 000000000027f6ec /system/framework/framework.jar (offset 0x2430000) (com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run+0) #78 pc 00000000005ef3f2 /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.__uniq.112435418011751916792819755956732575238)+466) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #79 pc 00000000009978fb /apex/com.android.art/lib64/libart.so (artQuickToInterpreterBridge+907) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #80 pc 000000000022c99c /apex/com.android.art/lib64/libart.so (art_quick_to_interpreter_bridge+140) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #81 pc 00000000007fc865 /system/framework/x86_64/boot-framework.oat (com.android.internal.os.ZygoteInit.main+2693) (BuildId: 5539d75c58ae9a874c8bdd56a20d5ad37b7a2757) #82 pc 0000000000212136 /apex/com.android.art/lib64/libart.so (art_quick_invoke_static_stub+806) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #83 pc 000000000046175c /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+252) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #84 pc 000000000088bc46 /apex/com.android.art/lib64/libart.so (art::JValue art::InvokeWithVarArgs<_jmethodID*>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, __va_list_tag*)+518) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #85 pc 000000000072f31a /apex/com.android.art/lib64/libart.so (art::JNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*, __va_list_tag*)+602) (BuildId: 53a0485d03b4e7011d7d555e293a17af) #86 pc 00000000000eefae /system/lib64/libandroid_runtime.so (_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+126) (BuildId: 0e1c658a58fcbbaafa02d3d78a368ee8) #87 pc 000000000011b496 /system/lib64/libandroid_runtime.so (android::AndroidRuntime::start(char const*, android::Vectorandroid::String8 const&, bool)+966) (BuildId: 0e1c658a58fcbbaafa02d3d78a368ee8) #88 pc 0000000000004708 /system/bin/app_process64 (main+1640) (BuildId: e2134dffba60b5f40425e9c838419a02) #89 pc 0000000000054ea5 /apex/com.android.runtime/lib64/bionic/libc.so (__libc_init+117) (BuildId: 2fa253bed5a3b3aee8fce00edb55a139) Lost connection to device.
These are the first layers of the .param file and the last ones.
7767517 275 327 Input in0 0 1 in0 Convolution conv_0 1 1 in0 1 0=32 1=3 11=3 12=1 13=2 14=1 2=1 3=2 4=1 5=1 6=864 Swish silu_84 1 1 1 2 Convolution conv_1 1 1 2 3 0=64 1=3 11=3 12=1 13=2 14=1 2=1 3=2 4=1 5=1 6=18432 Swish silu_85 1 1 3 4 Convolution conv_2 1 1 4 5 0=64 1=1 11=1 12=1 13=1 14=0 2=1 3=1 4=0 5=1 6=4096
Last layer
Split splitncnn_27 1 2 313 314 315 BinaryOp add_16 2 1 310 312 316 0=0 Split splitncnn_28 1 2 316 317 318 BinaryOp add_17 2 1 314 317 319 0=0 BinaryOp div_18 1 1 319 320 0=3 1=1 2=2.000000e+00 BinaryOp sub_19 2 1 318 315 321 0=1 Concat cat_21 2 1 320 321 322 0=0 Reshape reshape_167 1 1 252 323 0=8400 1=1 BinaryOp mul_20 2 1 322 323 324 0=2 Sigmoid sigmoid_163 1 1 303 325 Concat cat_22 2 1 324 325 out0 0=0
I thank you in advance for your attention and help.
UltralyticsAssistantUltralytics
👋 Hello, thank you for your interest in Ultralytics 🚀! It's exciting to see discussions around optimizing YOLO models for mobile and embedded devices. The link you shared regarding NCNN integration is a great starting point for exploring this topic: https://docs.ultralytics.com/zh/integrations/ncnn/.
If you're encountering issues or have specific queries regarding the NCNN export or deployment, here are a few recommendations:
-
If this is a 🐛 Bug Report, please provide a minimum reproducible example so our team can better understand and debug any potential issues.
-
If this is an ❓ integration or optimization question, please provide as much context as possible, such as your model configuration, device specifications, and any logs or error messages.
For further learning and exploration, you may also find these resources helpful:
- Integration Examples: Comprehensive guides for integrating YOLO with various platforms and tools.
- NCNN Documentation: Specific details about NCNN and its benefits for resource-constrained environments.
Upgrade
Ensure you're using the latest Ultralytics package and dependencies by upgrading your environment with the following command:
pip install -U ultralyticsEnvironments
To experiment with YOLO in diverse, GPU-enabled environments, check out the following setups:
- Notebooks with free GPU:
- Cloud services like Google Cloud and AWS.
- Docker Image for efficient deployment and reproducibility
.
Status
This badge indicates the status of Ultralytics CI tests, which ensure the reliability of YOLO models across various environments. A green badge means all tests are successful ✅.
Lastly, this is an automated response to guide you as quickly as possible, but rest assured that an Ultralytics engineer will follow up soon to assist you further. Feel free to share additional information or questions in the meantime 😊.
haitao5676I have created a single target tracking model for deployment on Android devices. How can I export the ncnn file for deployment using my own model?
haitao5676I have resolved the issue of using the model trained by ncnn in Python and am now preparing to deploy Java references for Android. Excuse me!
Glenn JocherUltralytics@xiao5676wen thank you for your update. To deploy your YOLO11 NCNN model on Android, you'll need to build NCNN for Android using Tencent NCNN's build guide and integrate the exported model using the NCNN Android SDK. For detailed integration steps, refer to the official NCNN documentation for Android. Let me know if you encounter issues!
-
haitao5676How can I export ncnn using the PT file I trained myself?
Glenn JocherUltralytics@xiao5676wen to export your trained YOLO11
.ptmodel to NCNN format, load your model using theYOLOclass in Python and then call theexportmethod withformat="ncnn". Example:from ultralytics import YOLO model = YOLO("path_to_your_trained_model.pt") model.export(format="ncnn") # Creates the NCNN modelFor additional details, refer to the NCNN export documentation.
trungnb34
Hi everyone NCNN can be inferred from the int8 model I have a question. when converting Yolo model to int8 . Can ultralytics support it and Does the accuracy decrease much? Thanks
Glenn JocherUltralytics(已编辑)@trungnb34 Ultralytics can export YOLO models to NCNN in FP16 with
quantize=16, but INT8 quantization is not currently supported for NCNN exports (quantize=8is rejected for this format). For details, refer to the Export to NCNN guide.robert-hardman
Hello ultralytics==8.3.53 Like it doesn't support int8 yet
File "C:\Users***\AppData\Local\Programs\Python\Python312\Lib\site-packages\ultralytics\engine\exporter.py", line 253, in call validate_args(fmt, self.args, fmts_dict["Arguments"][flags.index(True) + 1]) File "C:\Users***AppData\Local\Programs\Python\Python312\Lib\site-packages\ultralytics\engine\exporter.py", line 144, in validate_args assert arg in valid_args, f"ERROR \u274c\ufe0f argument '{arg}' is not supported for format='{format}'" ^^^^^^^^^^^^^^^^^ AssertionError: ERROR \u274c\ufe0f argument 'int8' is not supported for format='ncnn'
Glenn JocherUltralytics@robert-hardman thank you for raising this issue. Currently, INT8 quantization is not supported for NCNN exports in Ultralytics. For updates on supported features, keep an eye on our Export documentation. Let us know if you have further questions!
1
amrboef66
Hellooo, im trying to run interference on a raspberry Pi 5, now indeed NCNN has the fastest time, but i need it to be even quicker, so i tried reducing the image format from 640 to 480, since my model does not require such a large image size, but when i do that it misidentifies and puts bounding boxes everywhere on screen. How do i reduce image size while still allowing the model to run accurately?
Also, do you have any thoughts on how i can make it quicker, perhaps a different method? I am already using Nano model and int8 quantisation.
Thank you so much!
Glenn JocherUltralytics@amrboef66 to improve accuracy with a smaller image size, consider fine-tuning your model on images of the desired size. Additionally, you might explore using a more efficient model architecture or further optimizing your inference pipeline.
abdullah000123
Can we directly export model in ncnn while training or we only can convert model which i obtained after training ie converting from pt to ncnn if we can convert directly while training whats is benefit in case of accuracy and also give me code if we can export directly while training model
Paula DerrengerUltralytics@abdullah000123 you can only export a YOLOv8 model to NCNN format after training is complete. Direct export during training is not supported. Exporting post-training ensures that the model is fully trained and optimized before conversion, which does not impact accuracy. For exporting, you can use the following command after training:
from ultralytics import YOLO model = YOLO("yolov8n.pt") model.export(format="ncnn")For more details, please refer to the NCNN export guide.
woerstcase
Hey there!
I tried and on my Mac M2 Mac the inference time got from around 60ms to 30ms. And in a custom Model even faster.
But now i have the "problem" that the BBoxes are very flickering. Is there a good technique to make the bbox drawing even more smooth. I tried some approaches like drawing over 5 Frames for e.g. But if i have same classes on the image the bbox starts wandering trough the image.
best regards
Glenn JocherUltralyticsHey there! 👋
Great to hear about the speed improvements on your Mac M2! For smoothing out the flickering BBoxes, one approach is to apply a moving average or some form of temporal smoothing to the bounding box coordinates across frames. This helps in making the position more stable. A simple example could be:
# Assuming bbox is a list of bounding box coordinates for the current frame # prev_bboxes is a list of previous bounding boxes (could store N previous states) smoothed_bbox = current_bbox * alpha + prev_bbox * (1 - alpha)alphais a weighting factor that determines the balance between the current and previous frame's information (e.g.,alpha=0.6). This approach assumesprev_bboxis the bounding box coordinate from the previous frame. Adjustalphabased on your preference for smoothness vs. responsiveness.If you're handling multiple objects of the same class, consider incorporating an object tracker that assigns unique IDs to each object. This way, you can apply the smoothing individually to each tracked object, preventing the "wandering" effect across objects of the same class.
Hope this helps! Let us know if you have any more questions. 😊
Best regards
1woerstcase
Worked for me! With my custom Model the inference time is around 15ms!
This is my smoothing function:
#Smoothing function def smooth_bbox(current_bbox, prev_bbox): if prev_bbox is None: return current_bbox smoothed_bbox = [(c * alpha + p * (1 - alpha)) for c, p in zip(current_bbox, prev_bbox)] return smoothed_bbox
....#later in the code i use it like this
for box, confidence, cls in zip(boxes, confidences, classes): x1, y1, x2, y2 = map(int, box[:4]) # Extract bounding box coordinates class_name = names[int(cls)] # Generate a unique key for each bbox assuming no tracking ID available bbox_key = f"{x1}_{y1}_{x2}_{y2}" # Apply smoothing current_bbox = [x1, y1, x2, y2] if bbox_key in prev_bboxes: smoothed_coords = smooth_bbox(current_bbox, prev_bboxes[bbox_key]) else: smoothed_coords = current_bbox # Update prev_bboxes prev_bboxes[bbox_key] = smoothed_coords # Calculate the center of the smoothed bbox center_x = int((smoothed_coords[0] + smoothed_coords[2]) / 2) center_y = int((smoothed_coords[1] + smoothed_coords[3]) / 2)best regards
Glenn JocherUltralyticsHey there!
Great to see your impressive inference times, and thanks for sharing your smoothing function! 🚀 It looks like a solid approach. To address the flickering and "wandering" of bounding boxes for objects of the same class, you might consider integrating an object tracking algorithm like SORT or DeepSORT that assigns unique IDs to each object. This way, you can ensure that the smoothing you apply is object-specific.
Here's a small tweak to include object tracking by simply creating a unique key based on the object's class and center coordinates, assuming you'd implement tracking to assign IDs:
# Assuming bbox_key generation after object tracking assigns a unique track_id bbox_key = f"{class_name}_{track_id}" # Use this key for previous bbox retrieval and smoothing applicationThis tweak can help keep the smoothing consistent across consecutive frames for the same object, reducing "wandering."
Hope this helps make your bboxes smooth as silk! Let me know if you have other questions or if there's more we can help with.
Happy coding! 😊
Best regards