Intel OpenVINO 导出
OpenVINO 是 Intel 推出的开源工具包,用于优化和运行 AI 推理。使用 format="openvino" 导出 Ultralytics YOLO26 模型会创建一个 yolo26n_openvino_model/ 目录,其中包含 OpenVINO IR 模型(.xml 和 .bin 文件),可用于 Intel CPU、Intel 集成和独立 GPU,以及 Intel NPU,精度支持 FP32、FP16 或 INT8。
概览#
| 属性 | 详情 |
|---|---|
format 参数 | openvino |
| 输出 | yolo26n_openvino_model/,一个包含 yolo26n.xml、yolo26n.bin 和 metadata.yaml 的目录;yolo26n_int8_openvino_model/,用于 quantize=8 导出 |
| 运行平台 | Intel CPU、Intel 集成和独立 GPU,以及 Intel NPU。CPU 设备也可在其他 x86-64 和 ARM64 处理器上运行,例如 AMD 和 Apple silicon |
| 运行时 | OpenVINO Runtime 2024.0.0 或更高版本(openvino Python 包) |
| 精确率 | FP32、FP16、INT8。YOLO26n 的 FP32 模型为 10.1 MB,FP16 模型为 5.3 MB,INT8 模型为 3.3 MB |
| 导出主机 | Linux、macOS 或 Windows。导出在 CPU 上运行,无需 GPU |
Ultralytics Platform 可以帮你将训练好的模型导出为 OpenVINO 格式,因此你无需准备本地导出环境。
快速入门#
安装带有 OpenVINO 导出依赖项的 Ultralytics 包:
pip install "ultralytics[export-openvino]"这会安装 openvino,并且在 Python 3.9 或更高版本上安装用于 INT8 量化的 nncf。使用普通的 pip install ultralytics 时,首次导出会安装缺少的软件包。导出对版本有以下要求:
| 软件包 | 所需版本 |
|---|---|
torch | 2.1 或更高版本 |
openvino | 2024.0.0 或更高版本;在 macOS 15.4 及更高版本上需为 2025.2.0 或更高版本 |
nncf | 2.14.0 或更高版本,仅用于 quantize=8;低于 3.0.0,且 PyTorch 低于 2.3 |
其他安装方式请参阅安装指南。然后导出模型并加载导出的目录,以运行推理或验证准确率。在 CPU 上使用 OpenVINO 的课程会带你完成相同的步骤。
from ultralytics import YOLO
# 加载 YOLO26 模型
model = YOLO("yolo26n.pt")
# 将模型导出为 OpenVINO 格式
model.export(format="openvino") # creates 'yolo26n_openvino_model/'
# 使用校准数据导出经过 INT8 量化的 OpenVINO 模型
model.export(format="openvino", quantize=8, data="coco8.yaml") # creates 'yolo26n_int8_openvino_model/'from ultralytics import YOLO
# 加载导出的 OpenVINO 模型
model = YOLO("yolo26n_openvino_model/")
# 运行推理
results = model("https://ultralytics.com/images/bus.jpg")
# 在 Intel GPU 上运行推理;使用 "npu" 指定 Intel NPU,或使用 "cpu" 指定 CPU
results = model("https://ultralytics.com/images/bus.jpg", device="xpu")from ultralytics import YOLO
# 加载导出的 OpenVINO 模型
model = YOLO("yolo26n_openvino_model/")
# 在 COCO8 数据集上验证准确率
metrics = model.val(data="coco8.yaml")支持的任务#
OpenVINO 导出支持 Ultralytics 的全部七种任务,以及 YOLOv8、YOLO11 和 YOLO26 模型:检测、实例分割、语义分割、深度估计、分类、姿态估计和 OBB。语义分割和深度估计仅适用于 YOLO26,因为只有 YOLO26 系列提供这两种检测头。
导出参数#
| 参数 | 类型 | 默认值 | 说明 |
|---|---|---|---|
format | str | 'openvino' | 导出模型的目标格式,用于定义与各种部署环境的兼容性。 |
imgsz | int 或 tuple | 640 | 模型输入的图像尺寸:正方形图像使用整数,指定尺寸使用元组 (height, width)。如果未传入,导出时会使用检查点训练时的尺寸。官方 YOLO26 检查点中,检测、实例分割、姿态估计模型以及基于 ADE20K 预训练的语义模型使用 640,OBB 模型和基于 Cityscapes 预训练的语义模型使用 1024,深度模型使用 768,分类模型使用 224。 |
quantize | int 或 str | None | 量化精度:16(FP16:权重以半精度存储,输入和输出保持 FP32)或 8(使用 NNCF 进行 INT8 训练后量化,校准图像来自 data,输出到 _int8_openvino_model/ 目录);32/未设置表示 FP32。使用 quantize=8 训练的检查点无法导出为 OpenVINO 格式,参阅限制和故障排除。此选项取代了已弃用的 half/int8 标志。 |
data | str | None | 用于 INT8 校准的数据集 YAML;分类任务则使用数据集目录或内置数据集名称。除非设置了 split,否则校准会使用 val 划分。如果使用 quantize=8 时未指定该参数,Ultralytics 会使用适用于模型任务的小型默认数据集,例如用于检测的 coco8.yaml。 |
fraction | float、int 或 list | 1.0 | 校准子集,可按比例、图像数量或 [train, val, test] 比例/数量指定。包含两个项目的列表会让 test 保持完整,而 0 会跳过它。 |
dynamic | bool | False | 使输入的批次、高度和宽度变为动态,让单个模型能够接受不同的批次大小和图像尺寸。在 Intel NPU 上,动态模型仍会以导出时的 imgsz 运行,参阅部署。 |
nms | bool,可选 | None | 选择原始输出(None,默认)、嵌入式 NMS(True)或无 NMS 头(False)。请参阅输出格式。 |
batch | int | 1 | 导出模型的批次大小。静态导出会在输入形状中固定此批次大小。该值也是 INT8 校准使用的批次大小。 |
device | str | None | 导出始终在 CPU 上运行。使用 quantize=8 时,device=npu 会为 YOLO 检测、分割、姿态和 OBB 模型选择对称 INT8 激活量化,使这些模型在 Intel NPU 上的运行速度最高可提升 2 倍。 |
所有格式共有的参数请参阅导出模式文档。
INT8 量化#
quantize=8 使用 NNCF 进行训练后量化:导出过程会将校准图像输入模型,并尽可能将权重和激活存储为 8 位整数。
- 校准数据:导出过程会使用
data的val划分中的每个完整批次进行校准,也可以使用你通过split设置的划分。使用fraction限制图像数量。图像少于 300 张时,导出过程会发出警告。 - 检测头:对于带有 YOLO 检测头的模型(检测、分割、姿态和 OBB)以及 RT-DETR,用于解码检测头输出的操作(重塑、拼接、框解码和 DFL)以及所有 sigmoid 操作都保持浮点精度,而检测头的卷积则进行量化。分类、语义分割和深度模型没有排除的操作。
- 输入和输出:INT8 模型保留 FP32 模型的
float32输入和输出,因此为 FP32 模型编写的推理代码无需修改即可运行 INT8 模型。 - Intel NPU:如果计划在 Intel NPU 上运行 INT8 导出模型,请添加
device=npu。它会将 YOLO 检测、分割、姿态和 OBB 模型的激活量化方式从非对称切换为对称。
from ultralytics import YOLO
model = YOLO("yolo26n.pt")
# 使用 COCO 验证图像的 10%(500 张图像)进行校准;将 coco.yaml 替换为你自己的数据集
model.export(format="openvino", quantize=8, data="coco.yaml", fraction=0.1)
# 加载导出的 INT8 模型并运行推理
model = YOLO("yolo26n_int8_openvino_model/")
results = model("https://ultralytics.com/images/bus.jpg")在基准测试中,INT8 模型在服务器 CPU 上的运行速度是 FP32 的 1.02 到 1.33 倍,在 COCO val2017 上的 mAP 下降了 0.5 到 0.9。在 Intel Core Ultra CPU 上,INT8 的运行速度是 FP32 的 1.2 到 3.8 倍。部署前,请使用你自己的数据验证 INT8 模型。
输出格式#
导出会写入一个包含三个文件的目录:
| 文件 | 内容 |
|---|---|
yolo26n.xml | OpenVINO IR 格式的网络图 |
yolo26n.bin | 权重 |
metadata.yaml | 类别名称、任务、图像尺寸、批次大小、步幅和导出参数;Ultralytics 加载模型时会读取这些信息 |
模型接收一个形状为 (batch, 3, height, width) 的输入,采用 RGB 格式且缩放到 0-1。FP32、FP16 和 INT8 模型的输入和输出均为 float32,但语义分割输出除外:该输出是 uint8 类别图(类别数超过 256 时为 int32)。张量不带有 ONNX 导出中的 images 和 output0 名称,因此在你自己的代码中应通过索引访问输入和输出。
输出形状与 ONNX 导出相同,取决于任务和 nms 参数:在 imgsz=640 导出的 YOLO26n 检测模型,使用默认的 nms=None 时会返回 (1, 84, 8400),使用 nms=False 或 nms=True 时会返回 (1, 300, 6)。各任务输出的形状和内容请参阅 ONNX 输出格式。使用 nms=True 时,置信度阈值 conf(默认值 0.25)、IoU 阈值 iou、agnostic_nms 和 max_det 会在导出时固定到计算图中。在预测或验证时降低 conf,无法恢复计算图已移除的检测结果;要更改这些值,请重新导出。
部署#
YOLO("yolo26n_openvino_model/") 使用 OpenVINO Runtime 运行模型,并将 device 参数映射到 OpenVINO 设备:
device | OpenVINO 设备 |
|---|---|
| 未设置 | AUTO,让 OpenVINO 在 CPU 和 Intel GPU 之间选择(其默认优先级列表不包括 NPU);如果 OpenVINO 只找到 CPU,则使用 CPU |
cpu | CPU |
xpu 或 xpu:0 | GPU 或 GPU.0,即 Intel 集成或独立 GPU。在有两块 Intel GPU 的机器上,xpu:1 会选择 GPU.1 |
npu | NPU,即 Intel Core Ultra 处理器的神经处理单元 |
其他取值(例如 0 或 mps)会按未设置 device 时的方式处理。较早的 intel:cpu、intel:gpu 和 intel:npu 名称仍然可用,但会记录弃用警告。推理设备表会比较这些名称在不同格式下的对应情况。日志行 Using OpenVINO LATENCY mode for batch=1 inference on CPU... 会显示模型加载时 OpenVINO 报告的设备。使用 AUTO 时,这里可能显示 CPU,而 Intel GPU 仍在编译模型,因此请设置 device 以确保选择正确。对于 .pt 模型,device="xpu" 会改为选择 PyTorch Intel GPU 后端,该后端也是在 Intel GPU 上训练时使用的设备;而 device="npu" 会选择华为昇腾 NPU,而不是 Intel NPU。
模型在推理时的行为取决于导出方式:
- 静态导出(默认):模型始终以导出时的
imgsz和batch运行;Ultralytics 会从metadata.yaml中读取这些值。传递给predict的imgsz会被忽略。较大的批次会拆分为多个导出批次大小的块,较小的批次则会填充,因此可以处理任意数量的图像。 - 动态导出(
dynamic=True):predict可以接受不同的批次大小,以及任何为模型步幅倍数的imgsz。Intel NPU 和 Intel AMX CPU 上的 INT8 模型是例外,参阅限制和故障排除。 - 低延迟模式:Ultralytics 使用 OpenVINO
LATENCY性能提示编译模型,并同步运行推理。若要使用THROUGHPUT提示处理许多并行请求,请按照延迟和吞吐量指南中的示例使用 OpenVINO Runtime API。 - Intel NPU:Ultralytics 会针对 NPU 上的批次大小 1 编译模型,并将更大的批次拆分为单张图像。在 NPU 上,动态导出会固定为导出时的
imgsz。
使用 OpenVINO Runtime API 运行#
若要在 Ultralytics 包之外运行模型,请使用 openvino 包加载 .xml 文件。模型需要预处理后的图像,并返回输出格式中所述的原始输出,因此,填充到矩形画布以及在默认导出时执行 NMS 都需要由你的代码负责:
# 首先导出:yolo export model=yolo26n.pt format=openvino nms=False
import numpy as np
import openvino as ov
import yaml
core = ov.Core()
model = core.read_model("yolo26n_openvino_model/yolo26n.xml")
compiled = core.compile_model(model, "AUTO") # 或 "CPU"、"GPU"、"NPU"
image = np.zeros((1, 3, 640, 640), dtype=np.float32) # 你预处理后的图像:RGB、NCHW、0-1
detections = compiled(image)[0][0] # 第一个输出的第一张图像:形状为 (300, 6)
detections = detections[detections[:, 4] > 0.25] # 保留置信度高于阈值的行
with open("yolo26n_openvino_model/metadata.yaml") as f:
names = yaml.safe_load(f)["names"] # {0: 'person', 1: 'bicycle', ...}部署指南#
- OpenVINO 低延迟和高吞吐量模式:你可以选择针对单个数据流优化的最低延迟模式,或针对多个数据流优化的最高吞吐量模式。
- Intel DL Streamer:在 Intel Core Ultra CPU、GPU 和 NPU 上的 GStreamer 视频分析流水线中运行导出的模型。
- OpenVINO 推理设备:CPU、GPU 和 NPU 设备,以及 OpenVINO 文档中的
AUTO模式。 - OpenVINO 笔记本:Intel 的教程笔记本,包括 YOLO26 优化。
限制和故障排查#
- PyTorch 版本:在较旧的 PyTorch 版本上,导出会因
OpenVINO export requires torch>=2.1而失败。 - macOS 15.4 及更高版本:OpenVINO 2025.1.0 及更早版本在这些 macOS 版本上无法运行,因此在这些系统上导出需要
openvino>=2025.2.0,并会更新较旧的软件包。 - 使用
quantize=8训练的检查点:经过量化感知训练微调的检查点会因format='openvino' cannot export a QAT model: its Q/DQ ranges are only read by the 'onnx' and 'engine' backends而导出失败。请改用quantize=8导出微调前使用的检查点,或将微调后的检查点导出为 ONNX 或 TensorRT。 - 未指定
data的 INT8 导出:导出会记录INT8 export requires a missing 'data' arg for calibration. Using default 'data=coco8.yaml'.,并使用该任务的小型默认数据集进行校准。传入data,并使用能代表你部署数据的图像。 - INT8 校准规模:校准划分少于 300 张图像时,导出会记录
>300 images recommended for INT8 calibration, found 4 images.。coco8.yaml只有 4 张验证图像,足以测试导出,但不足以校准用于部署的模型。 - 输出目录:除 INT8 外,同一检查点的每次导出都会写入同一个
yolo26n_openvino_model/目录,无论其quantize、imgsz、dynamic、nms或batch是什么,因此后一次导出会覆盖前一次导出。所有 INT8 导出共用yolo26n_int8_openvino_model/,无论是否设置device=npu。 - 设备不可用:请求的设备不存在时,Ultralytics 会记录
OpenVINO device 'GPU' not available. Using 'AUTO' instead.(如果 OpenVINO 在机器上只找到 CPU,则记录Using 'CPU' instead.),并继续使用该设备运行。请确认你的处理器受 OpenVINO 支持,并安装 Intel GPU 或 NPU 驱动。 - Intel NPU:Ultralytics 在输入形状固定的情况下,以批次大小 1 在 NPU 上运行,参阅部署。传入
device=npu以使用 NPU:未设置device时不会选择 NPU。 - 带 NMS 的动态形状:使用
dynamic=True和nms=True导出时,如果batch保持为1,日志会记录'dynamic=True' export requires a maximum batch size, e.g. 'batch=16'.。Ultralytics 会将较大的批次拆分为多个导出时batch大小的块。 - Intel AMX CPU 上的动态 INT8 模型:在支持 AMX 的 CPU(Intel Sapphire Rapids 及更新型号)上运行 Linux 时,Ultralytics 会像运行静态模型一样,以导出时的
imgsz在CPU或AUTO设备上运行动态 INT8 模型:传递给predict或val的imgsz会被忽略,而val会使用导出时的batch。predict仍接受其他批次大小,并会针对每个新的批次大小重新编译模型,因此使用新批次大小进行首次推理时速度较慢。 - Windows:在 Windows 上,Ultralytics 会将 CPU 推理精度提示设为 FP32,从而阻止 OpenVINO 使用低精度浮点内核(INT8 模型仍以 INT8 运行),并将 oneDNN 限制为 AMX 以下的指令集。加载模型前设置
ONEDNN_MAX_CPU_ISA环境变量,即可使用其他限制。 - ARM64 上的 Linux:Ultralytics 会在 OpenVINO 精度执行模式下,以 FP32 推理精度运行
CPU设备。 - 内嵌 NMS:对于分类,
nms=True会触发'nms=True' is not valid for classification models.;对于语义分割和深度估计模型('nms=True' is not valid for semantic models. Forcing 'nms=None'.)以及 RT-DETR 模型('nms=True' is not available for end2end models or non-YOLO heads. Forcing 'nms=None'.),该选项会被忽略并发出警告。 - 无 NMS 头:
nms=False需要使用带有无 NMS 头的模型,例如 YOLO26 或 YOLOv10。对于其他模型(例如 YOLO11),导出时会发出This model has no one-to-one head; using one-to-many outputs.警告,并写入原始(1, 84, 8400)输出。
其他错误请参阅常见问题指南。
OpenVINO YOLO26 基准测试#
下表比较了 PyTorch 与每种 YOLO26 检测模型的 FP32、FP16 和 INT8 OpenVINO 模型。第一组数据是在服务器 CPU 上测得,准确率使用 COCO val2017;第二组数据是在三款 Intel Core Ultra 处理器上测得,涵盖它们的 CPU 和 GPU,以及其中两款处理器的 NPU。YOLO26 模型页面列出了每种模型的准确率、CPU ONNX 速度和 T4 TensorRT 速度。
服务器 CPU 上的 FP32、FP16 和 INT8#
本节中的所有数值均在一台配备两颗 AMD EPYC 9655 96 核处理器的服务器 CPU 上测得,而非 Intel CPU。PyTorch 使用 8 个线程,OpenVINO 使用 96 个线程,这是 Ultralytics 和 OpenVINO 在该服务器上选择的默认值,因此 PyTorch 与 OpenVINO 的运行时和线程数都有所不同。PyTorch 还使用矩形推理进行验证,即将每张图像填充为符合其宽高比的矩形(480x640 图像填充为 512x672);而静态 OpenVINO 模型始终以 640x640 运行。
每张图像的毫秒数,越低越好。每个模型最快的格式以粗体显示。
| 模型 | imgsz | PyTorch | OpenVINO FP32 | OpenVINO FP16 | OpenVINO INT8 |
|---|---|---|---|---|---|
| YOLO26n | 640 | 13.31 | 3.41 | 3.33 | 3.35 |
| YOLO26s | 640 | 24.28 | 5.18 | 5.18 | 4.51 |
| YOLO26m | 640 | 49.94 | 8.39 | 8.34 | 6.60 |
| YOLO26l | 640 | 63.75 | 10.92 | 11.05 | 8.94 |
| YOLO26x | 640 | 115.11 | 17.12 | 17.16 | 12.83 |
同一台服务器上的 8 个 CPU 核心运行 YOLO26n(PyTorch 和 OpenVINO 都通过 taskset 绑定到该服务器的 CPU 0-7):
| 指标 | PyTorch | OpenVINO FP32 | OpenVINO FP16 | OpenVINO INT8 |
|---|---|---|---|---|
| 延迟(毫秒/图像) | 9.28 | 7.13 | 7.14 | 6.71 |
| mAPval 50-95 | 40.8 | 40.9 | 40.9 | 40.3 |
- 在上述默认线程数下,OpenVINO FP32 的运行速度是 PyTorch 的 3.9 到 6.7 倍,准确率相差不超过 0.1 mAP。在绑定到相同的 8 个 CPU 核心时,PyTorch 本身的运行速度比默认配置更快;此时 OpenVINO FP32 运行 YOLO26n 的速度是 PyTorch 的 1.3 倍,INT8 的速度是 FP32 的 1.06 倍。
- FP16 将模型大小减半,准确率与 FP32 相差不超过 0.1 mAP,延迟与 FP32 相当(相差不超过 3%)。
- INT8 的运行速度是 FP32 的 1.02 到 1.33 倍,模型缩小至原来的 1/3.1 到 1/3.9,mAP 下降了 0.5 到 0.9。
测量使用了 ultralytics 8.4.212、OpenVINO 2026.4.1、NNCF 3.4.0 和 PyTorch 2.15.0,于 2026 年 11 月 14 日在 batch=1 上进行。延迟指 val 报告的推理阶段,不包括预处理或后处理。PyTorch、FP32 和 FP16 的数值取三次运行的平均值;运行间差异最高分别为 26% 和 6%。INT8 模型使用 591 张 COCO train2017 图像(split=train fraction=0.005)进行校准,并构建了三次;INT8 数值是三次构建的平均值。8 CPU 核心表格使用同一个 INT8 构建结果运行三次。
Intel Core Ultra CPU、GPU 和 NPU#
这些表格是在三款 Intel Core Ultra 处理器上测得,使用 openvino 2026.2.1 的基准测试模式:model.benchmark(data="coco128.yaml", format="openvino", device=...),并将 device 设为 cpu、xpu 或 npu。准确率是基于 128 张图像的 COCO128 数据集计算的 mAP50-95,因此不能与上文 COCO val2017 的数值进行比较。PyTorch 在同一处理器的 CPU 上运行。YOLO26 模型仅能在 Intel Core Ultra 2xxV 系列、3xx 系列及更新处理器的 NPU 上运行。
Intel Core Ultra X7 358H#
Intel Core Ultra X7 358H(Panther Lake)集成了 CPU 核心、Intel Arc 集成 GPU 和 Intel AI Boost NPU。
每张图像的毫秒数,越低越好。每种模型中速度最快的一列以粗体显示。
| 模型 | PyTorch CPU | CPU FP32 | CPU FP16 | CPU INT8 | GPU FP32 | GPU FP16 | GPU INT8 | NPU FP32 | NPU FP16 | NPU INT8 |
|---|---|---|---|---|---|---|---|---|---|---|
| YOLO26n | 29.28 | 14.35 | 14.23 | 9.99 | 4.09 | 4.03 | 4.24 | 7.68 | 7.71 | 8.60 |
| YOLO26s | 59.32 | 35.03 | 33.55 | 16.68 | 5.08 | 5.14 | 4.47 | 9.89 | 9.94 | 10.69 |
| YOLO26m | 143.21 | 90.53 | 88.82 | 33.33 | 7.10 | 7.06 | 5.61 | 13.02 | 13.13 | 14.84 |
| YOLO26l | 181.16 | 112.83 | 111.31 | 43.27 | 8.45 | 8.49 | 6.25 | 14.53 | 14.55 | 15.64 |
| YOLO26x | 379.75 | 241.21 | 236.87 | 78.92 | 13.03 | 12.98 | 8.74 | 23.48 | 23.41 | 21.56 |
更早一代的 Intel Core Ultra 处理器#
Intel Core Ultra 7 258V(Lunar Lake)
每张图像的毫秒数,越低越好。每种模型中速度最快的一列以粗体显示。
| 模型 | PyTorch CPU | CPU FP32 | CPU FP16 | CPU INT8 | GPU FP32 | GPU FP16 | GPU INT8 | NPU FP32 | NPU FP16 | NPU INT8 |
|---|---|---|---|---|---|---|---|---|---|---|
| YOLO26n | 27.33 | 16.84 | 16.73 | 8.62 | 3.33 | 3.34 | 3.46 | 23.71 | 23.70 | 23.96 |
| YOLO26s | 60.20 | 54.71 | 54.45 | 20.19 | 4.66 | 4.65 | 3.99 | 26.62 | 26.62 | 25.89 |
| YOLO26m | 152.87 | 178.59 | 179.25 | 53.07 | 9.03 | 8.98 | 5.70 | 32.94 | 33.01 | 30.95 |
| YOLO26l | 201.57 | 238.15 | 233.70 | 67.73 | 11.32 | 11.36 | 7.45 | 36.02 | 35.77 | 32.64 |
| YOLO26x | 431.04 | 567.22 | 561.39 | 151.11 | 19.08 | 19.63 | 13.69 | 49.64 | 49.69 | 41.75 |
Intel Core Ultra 7 155H(Meteor Lake)
每张图像的毫秒数,越低越好。每种模型中速度最快的一列以粗体显示。
| 模型 | PyTorch CPU | CPU FP32 | CPU FP16 | CPU INT8 | GPU FP32 | GPU FP16 | GPU INT8 |
|---|---|---|---|---|---|---|---|
| YOLO26n | 33.19 | 13.43 | 13.20 | 11.40 | 9.13 | 9.07 | 5.79 |
| YOLO26s | 70.92 | 36.16 | 36.07 | 15.54 | 16.37 | 16.65 | 9.27 |
| YOLO26m | 192.45 | 102.52 | 101.88 | 35.72 | 34.47 | 35.71 | 15.46 |
| YOLO26l | 239.19 | 129.24 | 128.50 | 45.38 | 43.24 | 43.40 | 19.72 |
| YOLO26x | 511.89 | 293.40 | 296.48 | 85.24 | 79.49 | 79.45 | 34.66 |
这三款处理器的表现:
- CPU:INT8 的运行速度是 FP32 的 1.2 到 3.8 倍,是 PyTorch 的 2.9 到 6.0 倍。X7 358H 和 155H 上的 FP32 运行速度是 PyTorch 的 1.6 到 2.5 倍,但在 258V 上运行 YOLO26m、YOLO26l 和 YOLO26x 时,速度低于 PyTorch。
- GPU:集成 GPU 上的 FP32 模型运行速度是 CPU 上 PyTorch 的 3.6 到 29.1 倍。除了 X7 358H 和 258V 上的 YOLO26n,所有模型的 INT8 速度都快于 FP32,最高可达 2.3 倍。
- NPU(X7 358H 和 258V):FP32 模型的运行速度是 CPU 上 PyTorch 的 1.2 到 16.2 倍,INT8 的运行速度则为 FP32 的 0.88 到 1.19 倍。这些 INT8 模型是在
device=npu选择对称激活之前导出的,因此表格中没有显示针对 NPU 调优的 INT8 导出模型。 - 所有设备上的 FP16 延迟与 FP32 相差不超过 5%,而 INT8 的 mAP50-95 下降了 0.1 到 2.7 个点。
关于 OpenVINO#
OpenVINO 是 Open Visual Inference and Neural Network Optimization(开放视觉推理与神经网络优化)的简称。它可以将模型从 PyTorch 等框架转换为自己的 IR 格式,并通过一个 API 在 Intel CPU、GPU 和 NPU 上运行模型。完整参考资料请参阅 OpenVINO 文档。
观看: 如何将 Ultralytics YOLO26 导出为 Intel OpenVINO 格式以加快推理速度 🚀
常见问题#
可以,在 CPU 上可以运行。OpenVINO 的
CPU设备支持 x86-64 和 ARM64 处理器:本页面的服务器 CPU 基准测试是在 AMD EPYC 处理器上测得的,同一导出模型也能在 Apple 芯片上运行。GPU和NPU设备需要 Intel 硬件,因此在其他机器上,device=xpu和device=npu会发出警告并回退。有关其他目标平台,请参阅模型部署选项指南。这取决于设备。FP16(
quantize=16)无需校准数据,模型大小减半,并且在基准测试中保持了 FP32 的准确率和延迟,因此如果模型大小很重要,可以选用 FP16。大多数模型在 INT8(quantize=8)下的 CPU 延迟最低:在服务器 CPU 上,YOLO26s 的延迟从每张图像 5.18 ms 降至 4.51 ms,mAPval 50-95 为 47.6,而 FP32 为 48.5;在 Intel Core Ultra CPU 上,INT8 的运行速度是 FP32 的 1.2 到 3.8 倍。在 Intel 集成 GPU 上,INT8 带来的提升较小,最高为 2.3 倍;在 NPU 上,本次测试中未使用device=npu导出的 INT8 模型速度与 FP32 大致相当。INT8 需要有代表性的校准数据,因此在选择 INT8 模型之前,请先用自己的数据验证模型。使用
dynamic=True导出,并在预测时传入所需的批次大小和imgsz。图像尺寸必须是模型步幅的倍数;除分类模型外,官方 YOLO26 检查点的步幅为 32。例外情况请参阅部署。动态导出from ultralytics import YOLO model = YOLO("yolo26n.pt") model.export(format="openvino", dynamic=True) model = YOLO("yolo26n_openvino_model/") results = model(["https://ultralytics.com/images/bus.jpg"] * 4, imgsz=320) # 一批 4 张图像,尺寸为 320 像素如果部署硬件是 Intel,请使用 OpenVINO:
device=xpu和device=npu分别选择 Intel GPU 和 NPU;在 Intel Core Ultra CPU 上,基准测试中的 INT8 OpenVINO 模型速度是 FP32 的 1.2 到 3.8 倍(见基准测试)。如果同一个文件还必须在 NVIDIA 或 AMD GPU 上运行,或在 OpenVINO 以外的运行时中运行,请使用 ONNX;在 Windows 上,Windows ML 可以为 ONNX 模型选择 Intel 硬件。
评论23
提出问题、分享结果,或帮助其他人。
Heyujie96
how can i export a trained cls or obb kind model to openvino kind
EduardoNew
I encountered some difficulties while trying to run the official video demo. Here is my code:
from ultralytics import YOLO
def main(): # Load a YOLOv8n PyTorch model model = YOLO("yolov8n.pt")
# Export the model model.export(format="openvino") # creates 'yolov8n_openvino_model/' # Load the exported OpenVINO model with explicit task definition ov_model = YOLO("yolov8n_openvino_model\\", task='detect') # Explicitly define task as 'detect' # Run inference results_original = model("D:\\python_YOLO_test\\bus.jpg") # Run ov inference results_ov = ov_model("D:\\python_YOLO_test\\bus.jpg")if name == "main": main() Additional notes: My environment is openvino==2024.0.0, and I executed pip install --upgrade torch ultralytics.
The issue I encountered is as follows:
Results saved to D:\python_YOLO_test Ultralytics YOLOv8.2.57 🚀 Python-3.11.9 torch-2.1.1+cu118 CPU (11th Gen Intel Core(TM) i7-11800H 2.30GHz) YOLOv8n summary (fused): 168 layers, 3,151,904 parameters, 0 gradients, 8.7 GFLOPs
PyTorch: starting from 'yolov8n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (6.2 MB)
OpenVINO: starting export with openvino 2024.0.0-14509-34caeefd078-releases/2024/0... OpenVINO: export success ✅ 2.8s, saved as 'yolov8n_openvino_model' (12.3 MB)
Export complete (5.2s) Results saved to D:\python_YOLO_test Predict: yolo predict task=detect model=yolov8n_openvino_model imgsz=640
Validate: yolo val task=detect model=yolov8n_openvino_model imgsz=640 data=coco.yaml
Visualize: https://netron.appimage 1/1 D:\python_YOLO_test\bus.jpg: 640x480 4 persons, 1 bus, 1 stop sign, 324.0ms Speed: 24.0ms preprocess, 324.0ms inference, 21.0ms postprocess per image at shape (1, 3, 640, 480) Loading yolov8n_openvino_model for OpenVINO inference... Using OpenVINO LATENCY mode for batch=1 inference...
Process finished with exit code -1073741819 (0xC0000005)
Process finished with exit code -1073740791 (0xC0000409) The model was successfully exported but did not perform inference, instead, it terminated directly. How should I resolve this issue?
Glenn JocherUltralytics(已编辑)@EduardoNew the exit code 0xC0000005 often indicates a memory access violation. Let's address this step-by-step:
-
Path Format: Try using forward slashes in your OpenVINO model path:
"yolov8n_openvino_model/"instead of backslashes. -
OpenVINO Version: Ultralytics requires
openvino>=2024.0.0. Try upgrading to a recent release:
pip install -U openvino- Hardware Compatibility: Ensure you have the latest Intel drivers installed from Intel's official site.
If issues persist, please create a Minimum Reproducible Example and open a GitHub Issue following our bug report guidelines. We'll be happy to investigate further!
-
Ranjith3436
You mentioned that the core CPU inference model takes 23.53 ms. I would like to know if the processing time decreases when we increase the number of CPU cores ? currently i am using 12 thread CPU.
Glenn JocherUltralytics@Ranjith3436 increasing the number of CPU cores can potentially reduce processing time, as it allows for parallel processing, but the actual speedup will depend on the model's ability to leverage multi-threading effectively.
Ranjith3436
how do i can know about model's ability?
Glenn JocherUltralyticsTo assess a model's ability to utilize multiple CPU cores, you can check its support for multi-threading or parallel processing in the documentation or by running benchmarks on your hardware configuration.
kimchi2020
hey i am working on object detection in videos and came across openvino that helps in increasing speed. how can i use openvino and yolov8 together to find objects in a video
Glenn JocherUltralytics@kimchi2020 hi there! It's great to hear that you're exploring OpenVINO to enhance the speed of your object detection tasks in videos. OpenVINO can indeed provide significant performance improvements, especially on Intel hardware.
To get started with using YOLOv8 and OpenVINO together, follow these steps:
Exporting YOLOv8 to OpenVINO Format
-
Export the Model: You can export your YOLOv8 model to the OpenVINO format using the following Python code:
from ultralytics import YOLO # Load a YOLOv8n PyTorch model model = YOLO("yolov8n.pt") # Export the model to OpenVINO format model.export(format="openvino") # creates 'yolov8n_openvino_model/'Alternatively, you can use the CLI:
yolo export model=yolov8n.pt format=openvino # creates 'yolov8n_openvino_model/' -
Run Inference with the Exported Model: After exporting, you can run inference using the exported OpenVINO model. Here’s how you can do it in Python:
from ultralytics import YOLO # Load the exported OpenVINO model ov_model = YOLO("yolov8n_openvino_model/") # Run inference results = ov_model("https://ultralytics.com/images/bus.jpg")Or using the CLI:
yolo predict model=yolov8n_openvino_model source='https://ultralytics.com/images/bus.jpg'
Running Inference on Videos
To process videos, you can modify the inference code to read frames from a video file and run the model on each frame. Here’s an example in Python:
import cv2 from ultralytics import YOLO # Load the exported OpenVINO model model = YOLO("yolov8n_openvino_model/") # Open the video file video_path = "path/to/your/video.mp4" cap = cv2.VideoCapture(video_path) # Loop through the video frames while cap.isOpened(): ret, frame = cap.read() if not ret: break # Run inference on the frame results = model(frame) # Visualize the results on the frame annotated_frame = results[0].plot() # Display the annotated frame cv2.imshow("YOLOv8 OpenVINO Inference", annotated_frame) # Break the loop if 'q' is pressed if cv2.waitKey(1) & 0xFF == ord('q'): break # Release the video capture object and close the display window cap.release() cv2.destroyAllWindows()Additional Resources
For more detailed information and advanced usage, you can refer to the OpenVINO documentation. This guide covers everything from exporting models to running inference and optimizing performance.
Feel free to reach out if you have any more questions. Happy coding! 🚀
-
kimchi2020
Traceback (most recent call last): File "/home/kimchi/Desktop/yolov8/onnx.py", line 18, in results = model(frame) ^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/model.py", line 174, in call return self.predict(source, stream, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/model.py", line 442, in predict return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/predictor.py", line 168, in call return list(self.stream_inference(source, model, *args, **kwargs)) # merge list of Result into one ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 35, in generator_context response = gen.send(None) ^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/predictor.py", line 234, in stream_inference self.model.warmup(imgsz=(1 if self.model.pt or self.model.triton else self.dataset.bs, 3, *self.imgsz)) File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/nn/autobackend.py", line 626, in warmup import torchvision # noqa (import here so torchvision import time not recorded in postprocess time) ^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/init.py", line 6, in from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/models/init.py", line 2, in from .convnext import * File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/models/convnext.py", line 8, in from ..ops.misc import Conv2dNormActivation, Permute File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/ops/init.py", line 1, in from ._register_onnx_ops import _register_custom_op File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/ops/_register_onnx_ops.py", line 5, in from torch.onnx import symbolic_opset11 as opset11 File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torch/onnx/init.py", line 59, in from ._internal.onnxruntime import ( File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torch/onnx/_internal/onnxruntime.py", line 36, in import onnx File "/home/kimchi/Desktop/yolov8/onnx.py", line 18, in results = model(frame) ^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/model.py", line 174, in call return self.predict(source, stream, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/model.py", line 442, in predict return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/predictor.py", line 168, in call return list(self.stream_inference(source, model, *args, **kwargs)) # merge list of Result into one ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 35, in generator_context response = gen.send(None) ^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/predictor.py", line 261, in stream_inference self.results = self.postprocess(preds, im, im0s) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/models/yolo/detect/predict.py", line 25, in postprocess preds = ops.non_max_suppression( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/utils/ops.py", line 291, in non_max_suppression i = torchvision.ops.nms(boxes, scores, iou_thres) # NMS ^^^^^^^^^^^^^^^ AttributeError: partially initialized module 'torchvision' has no attribute 'ops' (most likely due to a circular import). Did you mean: 'os'?
I am getting this error, how to get rid of it and what does it mean
Glenn JocherUltralyticsHi @kimchi2020,
Thank you for reaching out! The error you're encountering seems to be related to a circular import issue with the
torchvisionlibrary. This can happen if there's a naming conflict or an import loop in your code.Steps to Resolve the Issue:
-
Check for Naming Conflicts: Ensure that your script name (
onnx.py) is not conflicting with any library names. Rename your script to something unique, likerun_inference.py. -
Verify Installation: Make sure you have the latest versions of
torch,torchvision, andultralyticsinstalled. You can update them using:pip install --upgrade torch torchvision ultralytics -
Reproduce the Issue: If the issue persists, please provide a minimum reproducible example. This helps us diagnose the problem more effectively. You can refer to our guide on creating a reproducible example here: Minimum Reproducible Example.
Example Code for Inference with OpenVINO:
Here's a quick example of how to export your YOLOv8 model to OpenVINO and run inference on a video:
import cv2 from ultralytics import YOLO # Load a YOLOv8n PyTorch model model = YOLO("yolov8n.pt") # Export the model to OpenVINO format model.export(format="openvino") # creates 'yolov8n_openvino_model/' # Load the exported OpenVINO model ov_model = YOLO("yolov8n_openvino_model/") # Open the video file video_path = "path/to/your/video.mp4" cap = cv2.VideoCapture(video_path) # Loop through the video frames while cap.isOpened(): ret, frame = cap.read() if not ret: break # Run inference on the frame results = ov_model(frame) # Visualize the results on the frame annotated_frame = results[0].plot() # Display the annotated frame cv2.imshow("YOLOv8 OpenVINO Inference", annotated_frame) # Break the loop if 'q' is pressed if cv2.waitKey(1) & 0xFF == ord('q'): break # Release the video capture object and close the display window cap.release() cv2.destroyAllWindows()For more detailed information on exporting and using OpenVINO, please refer to our OpenVINO Integration Guide.
Feel free to reach out if you have any more questions or need further assistance. Happy coding! 🚀
-
YA12SHYAM
Hi @glenn-jocher , what if i want to use intel GPU's for the inference, using device=GPU doesn't seems to works , it looks for cuda i.e nvidia gpu
Glenn JocherUltralytics(已编辑)To use Intel GPUs for inference, export your YOLOv8 model to OpenVINO format with
model.export(format="openvino"), then run inference with the exported model and select the GPU withdevice="intel:gpu", for exampleov_model("bus.jpg", device="intel:gpu"). For more details, please refer to the OpenVINO integration guide.YA12SHYAM
ok, just to confirm i can't use ultralytics python api for inference with intel gpu like i usually do within the ultralytics ecosystem for example ?
from ultralytics import YOLO model = YOLO(path/to/model)here the path/to/model works with onnx, engine, opnevino;s xml , but it use cpu in case of openvino , i want to use my intel GPU
Glenn JocherUltralytics(已编辑)@YA12SHYAM to use Intel GPUs with YOLOv8, export your model to OpenVINO format and select the GPU at inference with
device="intel:gpu", e.g.model("bus.jpg", device="intel:gpu"). For detailed steps, refer to the OpenVINO integration guide.
kanishkasw
Hi Im getting the following error when attempting to convert the model to openVino. Tried with both 2024.1.0 and 2024.0.0. Im on ubuntu 20.04 with python 3.8 $ yolo export model=runs/classify/train/weights/best.pt format=openvino Ultralytics YOLOv8.2.31 🚀 Python-3.8.10 torch-2.3.1+cu121 CPU (Intel Core(TM) i9-7900X 3.30GHz) YOLOv8n-cls summary (fused): 73 layers, 1438723 parameters, 0 gradients, 3.3 GFLOPs
PyTorch: starting from 'runs/classify/train/weights/best.pt' with input shape (1, 3, 320, 320) BCHW and output shape(s) (1, 3) (2.8 MB) OpenVINO: export failure ❌ 0.0s: module 'openvino' has no attribute 'version' Traceback (most recent call last): File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/bin/yolo", line 8, in sys.exit(entrypoint()) File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/cfg/init.py", line 591, in entrypoint getattr(model, mode)(**overrides) # default args from model File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/model.py", line 602, in export return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model) File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/exporter.py", line 308, in call f[3], _ = self.export_openvino() File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/exporter.py", line 142, in outer_func raise e File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/exporter.py", line 137, in outer_func f, model = inner_func(*args, **kwargs) File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/exporter.py", line 454, in export_openvino LOGGER.info(f"\n{prefix} starting export with openvino {ov.version}...") AttributeError: module 'openvino' has no attribute 'version'
Paula DerrengerUltralytics(已编辑)@kanishkasw hi there,
Thank you for reaching out and providing detailed information about the issue you're encountering. It looks like you're experiencing an error when attempting to export your model to OpenVINO format. Let's work through this together.
First, let's ensure that you are using the latest versions of
torchandultralytics. Please run the following commands to update your packages:pip install --upgrade torch ultralyticsNext, it seems the error is related to the OpenVINO package. The error message
module 'openvino' has no attribute '__version__'suggests there might be an issue with the OpenVINO installation. Please make sure you have a recent version of OpenVINO installed (Ultralytics requiresopenvino>=2024.0.0). You can install it using the following command:pip install -U openvinoOnce you've updated the packages, try exporting the model again:
yolo export model=runs/classify/train/weights/best.pt format=openvinoIf the issue persists, could you please provide a minimum reproducible example? This will help us investigate the problem more effectively. You can find guidance on creating a minimum reproducible example here: Minimum Reproducible Example.
Additionally, here's a Python example to ensure everything is set up correctly:
from ultralytics import YOLO # Load a YOLOv8n PyTorch model model = YOLO("runs/classify/train/weights/best.pt") # Export the model to OpenVINO format model.export(format="openvino") # creates 'best_openvino_model/' # Load the exported OpenVINO model ov_model = YOLO("best_openvino_model/") # Run inference results = ov_model("https://ultralytics.com/images/bus.jpg")For more detailed instructions on using OpenVINO, please refer to our OpenVINO Integration Guide.
Feel free to reach out if you have any further questions or run into any other issues. We're here to help! 😊
renato-rodrig
hello everything is fine? I have a question about the results exposed in the article above. Is the time shown in the graphs in relation to the inference time that the model takes to return the detections or is it the time for image reading, image pre-processing and model inference? I ask this because on my computer (intel core i7 12 generation CPU) the yolov8n model ran with an inference time of 40ms and the gain from using OpenVINO was not that great.
Paula DerrengerUltralytics@renato-rodrig hello! 👋 Regarding your question about the times shown in the graphs, they typically refer to the total inference time, which includes the actual model inference as well as the time taken for pre-processing the input data (like image reading and resizing).
The difference you observed with OpenVINO on your Intel Core i7 may vary due to several factors such as the specific configuration of your system, the type of tasks being run, and the optimizations that the YOLOv8n model can leverage on that platform. Generally, OpenVINO tends to show more significant performance gains on integrated Intel GPUs and some specific CPU configurations.
If you haven't seen a major performance improvement, you might want to look into further tuning your OpenVINO deployment or explore using other optimization techniques provided by OpenVINO. Additionally, ensure all drivers and software components are up to date for optimal performance.
Here’s a quick example of ensuring you're using the model in the most optimized manner:
from ultralytics import YOLO # Load YOLOv8n PyTorch model model = YOLO('yolov8n.pt') # Export the model to OpenVINO model.export(format='openvino') # Creates 'yolov8n_openvino_model/' # Load the exported OpenVINO model ov_model = YOLO('yolov8n_openvino_model/') # Run inference to see if performance improves results = ov_model('https://ultralytics.com/images/bus.jpg')Feel free to experiment with different configurations and settings! 😊
1
YouROS12
I wanna do inferences with my exported pre-trained yolov8 model using Raspberry PI with a Neural Network Computing 2 (NCS2) plugged in.. but its doesn't support opset 11 or above.. only opset10.. I specified onnx opset to 10 and convert it to Openvino but it didn't work.
Anyway please to export openvino model directly in opset10 ? Does even yolov8 support this specified opset format ?
Thank you.
Glenn JocherUltralytics(已编辑)@YouROS12 the current OpenVINO export no longer goes through ONNX: Ultralytics converts the PyTorch model directly to OpenVINO IR with
openvino.convert_model, so the ONNXopsetargument does not apply and the oldmoModel Optimizer step isn't needed. Export with:from ultralytics import YOLO # Load a pretrained YOLOv8 model model = YOLO('yolov8n.pt') # Export the model to OpenVINO model.export(format='openvino') # creates 'yolov8n_openvino_model/'The export requires
openvino>=2024.0.0. If your target requires an older OpenVINO that cannot load these models, you can still trymodel.export(format='onnx', opset=10)and convert the ONNX file with that toolkit's own tools, but some YOLOv8 operations may not be supported in older opsets, which could lead to errors during export or inference.YouROS12
Where can I find documentation about YOLOv8's operations with opset 10 ? Updating OpenVINO toolkit is not an option because NCS2 works only at opset 10 and lower.
Paula DerrengerUltralytics@YouROS12 the documentation for ONNX opset versions and the operations they support can be found on the ONNX GitHub repository or the official ONNX documentation. However, there isn't specific documentation for YOLOv8's operations with opset 10, as this would be a combination of YOLOv8's implementation details and ONNX's opset specifications.
To check which operations are supported in opset 10, you can refer to the ONNX Operators documentation for that specific version. This documentation lists all the operators available in opset 10 and their details.
For YOLOv8, you would need to look at the model's architecture and identify which operations are used. Then, cross-reference these operations with the ONNX opset 10 documentation to see if they are supported. If certain operations are not supported in opset 10, you might need to modify the model or the export script to use alternative operations that are compatible with opset 10.
Since updating OpenVINO is not an option for you, if you find that YOLOv8 uses operations not supported in opset 10, you might need to consider using a different model architecture that is compatible with opset 10, or reach out to the community or Ultralytics for further assistance on how to proceed with your specific use case and hardware constraints.
