Ultralytics YOLO27: Join the waitlist.
Get Started

Intel OpenVINO Export

OpenVINO Intel AI inference toolkit

This guide covers exporting Ultralytics YOLO26 models to the OpenVINO format, which can provide up to 3x CPU speedup, as well as accelerating YOLO inference on Intel GPU and NPU hardware.

OpenVINO, short for Open Visual Inference & Neural Network Optimization toolkit, is Intel's toolkit for optimizing and deploying AI inference models. Even though the name contains Visual, OpenVINO also supports various additional tasks including language, audio, time series, etc.



Watch: How to Export Ultralytics YOLO26 to Intel OpenVINO Format for Faster Inference 🚀

Supported Tasks#

OpenVINO export supports all seven Ultralytics tasks. Semantic segmentation and depth estimation are available only with YOLO26, the only family that ships those heads.

TaskYOLOv8YOLO11YOLO26
Detect✅✅✅
Segment✅✅✅
Semantic❌❌✅
Depth❌❌✅
Classify✅✅✅
Pose✅✅✅
OBB✅✅✅

Installation#

Install the OpenVINO export dependencies with:

pip install "ultralytics[export-openvino]"

These dependencies are also included in ultralytics[export] and installed automatically when required by an export.

Usage Examples#

The OpenVINO format supports the Export, Predict, and Validate modes. Export your model, then load the exported model to run inference or validate its accuracy on Intel CPU, integrated/discrete GPU, or NPU.

Export
from ultralytics import YOLO

# Load a YOLO26 model
model = YOLO("yolo26n.pt")

# Export the model to OpenVINO format
model.export(format="openvino")  # creates 'yolo26n_openvino_model/'
Predict
from ultralytics import YOLO

# Load the exported OpenVINO model
model = YOLO("yolo26n_openvino_model/")

# Run inference
results = model("https://ultralytics.com/images/bus.jpg")

# Run inference on a specific device, available devices: ["intel:gpu", "intel:npu", "intel:cpu"]
results = model("https://ultralytics.com/images/bus.jpg", device="intel:gpu")
Validate
from ultralytics import YOLO

# Load the exported OpenVINO model
model = YOLO("yolo26n_openvino_model/")

# Validate accuracy on the COCO8 dataset
metrics = model.val(data="coco8.yaml")

Export Arguments#

ArgumentTypeDefaultDescription
formatstr'openvino'Target format for the exported model, defining compatibility with various deployment environments.
imgszint or tuple640Desired image size for the model input. Can be an integer for square images or a tuple (height, width) for specific dimensions.
quantizeint or strNoneQuantization precision: 16 (FP16) or 8 (INT8/PTQ; needs calibration data/fraction); 32/unset is FP32. Replaces the deprecated half/int8 flags.
dynamicboolFalseAllows dynamic input sizes, enhancing flexibility in handling varying image dimensions.
nmsbool, optionalNoneSelect raw output (None, default), embedded NMS (True), or the NMS-free head (False).
batchint1Specifies export model batch inference size or the max number of images the exported model will process concurrently in predict mode.
datastrNonePath to the dataset YAML, essential for quantization; classification instead takes a dataset directory or a built-in dataset name. If omitted with quantize=8, Ultralytics selects the default calibration dataset for the model task.
fractionfloat, int, or list1.0Calibration subset as a ratio, image count, or [train, val, test] ratios/counts. Two-item lists leave test full, while 0 skips it.
devicestrNoneSpecifies the device for exporting: CPU (device=cpu), MPS for Apple silicon (device=mps).

For more details about the export process, visit the Ultralytics documentation page on exporting.

Warning

OpenVINO™ is compatible with most Intel® processors, but to ensure optimal performance:

  1. Verify OpenVINO™ support Check whether your Intel® chip is officially supported by OpenVINO™ using Intel's compatibility list.

  2. Identify your accelerator Determine if your processor includes an integrated NPU (Neural Processing Unit) or integrated GPU by consulting Intel's hardware guide.

  3. Install the latest drivers If your chip supports an NPU or GPU but OpenVINO™ isn't detecting it, you may need to install or update the associated drivers. Follow the driver‑installation instructions to enable full acceleration.

By following these three steps, you can ensure OpenVINO™ runs optimally on your Intel® hardware.

Benefits of OpenVINO#

  1. Performance: OpenVINO delivers high-performance inference on Intel CPUs, integrated and discrete GPUs, and NPUs.
  2. Support for Heterogeneous Execution: OpenVINO provides an API to write once and deploy on any supported Intel hardware (CPU, GPU, NPU).
  3. Model Conversion: OpenVINO's model conversion API imports, converts, and optimizes models from popular deep learning frameworks such as PyTorch, TensorFlow, TensorFlow Lite, ONNX, and PaddlePaddle.
  4. Ease of Use: The toolkit comes with a large collection of tutorial notebooks (including YOLO26 optimization) teaching different aspects of the toolkit.

OpenVINO Export Structure#

When you export a model to OpenVINO format, it results in a directory containing the following:

  1. XML file: Describes the network topology.
  2. BIN file: Contains the weights and biases binary data.
  3. Metadata file: metadata.yaml holds model metadata such as class names, image size, and task, which Ultralytics reads when loading the model.

You can use these files to run inference with the OpenVINO Runtime.

Using OpenVINO Export in Deployment#

Once your model is successfully exported to the OpenVINO format, you have two primary options for running inference:

  1. Use the ultralytics package, which provides a high-level API and wraps the OpenVINO Runtime.

  2. Use the native openvino package for more advanced or customized control over inference behavior.

Inference with Ultralytics#

The ultralytics package runs inference on the exported OpenVINO model via the predict method. You can also specify the target device (e.g., intel:gpu, intel:npu, intel:cpu) using the device argument.

from ultralytics import YOLO

# Load the exported OpenVINO model
ov_model = YOLO("yolo26n_openvino_model/")  # the path of your exported OpenVINO model
# Run inference with the exported model
ov_model.predict(device="intel:gpu")  # specify the device you want to run inference on

This approach suits fast prototyping or deployment when you don't need full control over the inference pipeline.

Inference with OpenVINO Runtime#

The OpenVINO Runtime provides a unified API for inference across all supported Intel hardware. It also provides advanced capabilities like load balancing across Intel hardware and asynchronous execution.

Remember, you'll need the XML and BIN files as well as any application-specific settings like input size, scale factor for normalization, etc., to correctly set up and use the model with the Runtime.

In your deployment application, you would typically do the following steps:

  1. Initialize OpenVINO by creating core = Core().
  2. Load the model using the core.read_model() method.
  3. Compile the model using the core.compile_model() function.
  4. Prepare the input (image, text, audio, etc.).
  5. Run inference using compiled_model(input_data).

For more detailed steps and code snippets, refer to the OpenVINO documentation or API tutorial.

OpenVINO YOLO26 Benchmarks#

The Ultralytics team benchmarked YOLO26 across various model formats and precisions (FP32, FP16, and INT8), evaluating speed and accuracy on different Intel devices compatible with OpenVINO.

Note

  • The benchmarking results below are for reference and might vary based on the exact hardware and software configuration of a system, as well as the current workload of the system at the time the benchmarks are run.

  • All benchmarks were run with openvino Python package version 2026.2.1.

  • YOLO26 models on NPU are only supported on Intel® Core™ Ultra™ systems with 2xxV series and 3xx series and above.

Intel® Core™ Ultra#

The Intel® Core™ Ultra™ series combines CPU cores, an integrated GPU, and a dedicated Neural Processing Unit (NPU) in a single chip, so one system can handle traditional processing, GPU-accelerated workloads, and on-device AI inference.

This hybrid design lets OpenVINO run inference on the CPU, GPU, or NPU depending on the latency, throughput, and power needs of an application. The lineup ranges from energy-efficient designs to high-power variants marked by the "H" designation for laptops and compact form factors that need more compute.

Benchmarks below run on Intel® Core™ Ultra™ X7 358H, Intel® Core™ Ultra™ 7 258V and Intel® Core™ Ultra™ 7 155H at FP32, FP16 and INT8 precision.

Intel® Core™ Ultra™ X7 358H#

Benchmarks
YOLO26 OpenVINO benchmarks on the Intel Core Ultra X7 358H integrated Arc GPU
Detailed Benchmark Results
ModelFormatPrecisionStatusSize (MB)metrics/mAP50-95(B)Inference time (ms/im)
YOLO26nPyTorch (CPU)FP32✅5.30.476529.28
YOLO26nOpenVINOFP32✅9.70.47624.09
YOLO26nOpenVINOFP16✅5.10.47624.03
YOLO26nOpenVINOINT8✅3.20.46344.24
YOLO26sPyTorch (CPU)FP32✅19.50.570359.32
YOLO26sOpenVINOFP32✅36.70.56165.08
YOLO26sOpenVINOFP16✅18.70.56165.14
YOLO26sOpenVINOINT8✅10.00.54624.47
YOLO26mPyTorch (CPU)FP32✅42.20.6196143.21
YOLO26mOpenVINOFP32✅78.40.61667.1
YOLO26mOpenVINOFP16✅39.50.61667.06
YOLO26mOpenVINOINT8✅20.60.60555.61
YOLO26lPyTorch (CPU)FP32✅50.70.6215181.16
YOLO26lOpenVINOFP32✅95.30.62058.45
YOLO26lOpenVINOFP16✅48.10.62058.49
YOLO26lOpenVINOINT8✅25.20.59386.25
YOLO26xPyTorch (CPU)FP32✅113.20.6512379.75
YOLO26xOpenVINOFP32✅213.30.656813.03
YOLO26xOpenVINOFP16✅107.10.656812.98
YOLO26xOpenVINOINT8✅54.80.63858.74

Intel® Core™ Ultra™ 7 258V#

Benchmarks
YOLO26 OpenVINO benchmarks on the Intel Core Ultra 7 258V integrated Arc GPU
Detailed Benchmark Results
ModelFormatPrecisionStatusSize (MB)metrics/mAP50-95(B)Inference time (ms/im)
YOLO26nPyTorch (CPU)FP32✅5.30.476527.33
YOLO26nOpenVINOFP32✅9.60.47643.33
YOLO26nOpenVINOFP16✅5.10.47643.34
YOLO26nOpenVINOINT8✅3.20.46343.46
YOLO26sPyTorch (CPU)FP32✅19.50.570360.2
YOLO26sOpenVINOFP32✅36.70.56164.66
YOLO26sOpenVINOFP16✅18.60.56164.65
YOLO26sOpenVINOINT8✅10.00.54623.99
YOLO26mPyTorch (CPU)FP32✅42.20.6196152.87
YOLO26mOpenVINOFP32✅78.40.61689.03
YOLO26mOpenVINOFP16✅39.50.61688.98
YOLO26mOpenVINOINT8✅20.60.60555.7
YOLO26lPyTorch (CPU)FP32✅50.70.6215201.57
YOLO26lOpenVINOFP32✅95.30.620311.32
YOLO26lOpenVINOFP16✅48.10.620311.36
YOLO26lOpenVINOINT8✅25.20.59387.45
YOLO26xPyTorch (CPU)FP32✅113.20.6512431.04
YOLO26xOpenVINOFP32✅213.20.656819.08
YOLO26xOpenVINOFP16✅107.10.656819.63
YOLO26xOpenVINOINT8✅54.80.638513.69

Intel® Core™ Ultra™ 7 155H#

Benchmarks
YOLO26 OpenVINO benchmarks on the Intel Core Ultra 7 155H integrated Arc GPU
Detailed Benchmark Results
ModelFormatPrecisionStatusSize (MB)metrics/mAP50-95(B)Inference time (ms/im)
YOLO26nPyTorch (CPU)FP32✅5.30.476533.19
YOLO26nOpenVINOFP32✅9.60.47589.13
YOLO26nOpenVINOFP16✅5.10.47589.07
YOLO26nOpenVINOINT8✅3.20.45735.79
YOLO26sPyTorch (CPU)FP32✅19.50.570370.92
YOLO26sOpenVINOFP32✅36.70.561616.37
YOLO26sOpenVINOFP16✅18.60.561616.65
YOLO26sOpenVINOINT8✅10.00.5459.27
YOLO26mPyTorch (CPU)FP32✅42.20.6196192.45
YOLO26mOpenVINOFP32✅78.40.616734.47
YOLO26mOpenVINOFP16✅39.50.616735.71
YOLO26mOpenVINOINT8✅20.60.603915.46
YOLO26lPyTorch (CPU)FP32✅50.70.6215239.19
YOLO26lOpenVINOFP32✅95.30.62143.24
YOLO26lOpenVINOFP16✅48.10.62143.4
YOLO26lOpenVINOINT8✅25.20.599919.72
YOLO26xPyTorch (CPU)FP32✅113.20.6512511.89
YOLO26xOpenVINOFP32✅213.20.655279.49
YOLO26xOpenVINOFP16✅107.10.655279.45
YOLO26xOpenVINOINT8✅54.80.64234.66

Reproduce Our Results#

To reproduce the Ultralytics benchmarks above on all export formats, run this code:

Example
from ultralytics import YOLO

# Load a YOLO26n PyTorch model
model = YOLO("yolo26n.pt")

# Benchmark YOLO26n speed and accuracy on the COCO128 dataset for all export formats
results = model.benchmark(data="coco128.yaml")

Note that benchmarking results might vary based on the exact hardware and software configuration of a system, as well as the current workload of the system at the time the benchmarks are run. For the most reliable results, use a dataset with a large number of images, such as COCO (data='coco.yaml', 5000 val images).

Conclusion#

The benchmarking results show the benefits of exporting YOLO26 models to the OpenVINO format. Across model sizes and Intel hardware, OpenVINO runs inference much faster than PyTorch while maintaining comparable accuracy.

By converting models to the OpenVINO format, developers can achieve significant performance improvements on Intel CPUs, GPUs, and NPUs, making it easier to deploy these models in real-world applications.

For more detailed information and instructions on using OpenVINO, refer to the official OpenVINO documentation.

FAQ#

  • Exporting YOLO26 models to the OpenVINO format can significantly enhance CPU speed and enable GPU and NPU accelerations on Intel hardware. To export, you can use either Python or CLI as shown below:

    Example
    from ultralytics import YOLO

# Load a YOLO26n PyTorch model
model = YOLO("yolo26n.pt")

# Export the model
model.export(format="openvino")  # creates 'yolo26n_openvino_model/'

    For more information, refer to the export formats documentation.

  • Using Intel's OpenVINO toolkit with YOLO26 models offers several benefits:

    1. Performance: Achieve up to 3x speedup on CPU inference and use Intel GPUs and NPUs for acceleration.
    2. Model Conversion: Convert, optimize, and execute models from popular frameworks like PyTorch, TensorFlow, and ONNX.
    3. Ease of Use: A large collection of tutorial notebooks is available to help users get started, including ones for YOLO26.
    4. Heterogeneous Execution: Deploy models on various Intel hardware with a unified API.

    For detailed performance comparisons, visit our benchmarks section.

  • After exporting a YOLO26n model to OpenVINO format, you can run inference using Python or CLI:

    Example
    from ultralytics import YOLO

# Load the exported OpenVINO model
ov_model = YOLO("yolo26n_openvino_model/")

# Run inference
results = ov_model("https://ultralytics.com/images/bus.jpg")

    Refer to our predict mode documentation for more details.

  • Ultralytics YOLO26 is optimized for real-time object detection with high accuracy and speed. Specifically, when combined with OpenVINO, YOLO26 provides:

    • Up to 3x speedup on Intel CPUs
    • Deployment on Intel GPUs and NPUs
    • Consistent and comparable accuracy across various export formats

    For in-depth performance analysis, check our detailed YOLO26 benchmarks on different hardware.

  • Yes, you can benchmark YOLO26 models in various formats including PyTorch, TorchScript, ONNX, and OpenVINO. Use the following code snippet to run benchmarks on your chosen dataset:

    Example
    from ultralytics import YOLO

# Load a YOLO26n PyTorch model
model = YOLO("yolo26n.pt")

# Benchmark YOLO26n speed and accuracy on the COCO8 dataset for all export formats
results = model.benchmark(data="coco8.yaml")

    For detailed benchmark results, refer to our benchmarks section and export formats documentation.

Contributors
glenn-jocherglenn-jocher30lakshanthadlakshanthad11francesco-mttlfrancesco-mttl7raimbekovmraimbekovm4RizwanMunawarRizwanMunawar4Laughing-qLaughing-q2fcakyonfcakyon2GornokaGornoka1onuralpszronuralpszr1easyrider11easyrider111pderrengerpderrenger1MatthewNoyceMatthewNoyce1andrei-kochinandrei-kochin1abirami-vinaabirami-vina1

Comments23

Ask a question, share your results, or help someone else.

  2. EduardoNewEduardoNewGitHub

    I encountered some difficulties while trying to run the official video demo. Here is my code:

    from ultralytics import YOLO

    def main(): # Load a YOLOv8n PyTorch model model = YOLO("yolov8n.pt")

    # Export the model
model.export(format="openvino")  # creates 'yolov8n_openvino_model/'

# Load the exported OpenVINO model with explicit task definition
ov_model = YOLO("yolov8n_openvino_model\\", task='detect')  # Explicitly define task as 'detect'

# Run inference
results_original = model("D:\\python_YOLO_test\\bus.jpg")

# Run ov inference
results_ov = ov_model("D:\\python_YOLO_test\\bus.jpg")

    if name == "main": main() Additional notes: My environment is openvino==2024.0.0, and I executed pip install --upgrade torch ultralytics.

    The issue I encountered is as follows:

    Results saved to D:\python_YOLO_test Ultralytics YOLOv8.2.57 🚀 Python-3.11.9 torch-2.1.1+cu118 CPU (11th Gen Intel Core(TM) i7-11800H 2.30GHz) YOLOv8n summary (fused): 168 layers, 3,151,904 parameters, 0 gradients, 8.7 GFLOPs

    PyTorch: starting from 'yolov8n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (6.2 MB)

    OpenVINO: starting export with openvino 2024.0.0-14509-34caeefd078-releases/2024/0... OpenVINO: export success ✅ 2.8s, saved as 'yolov8n_openvino_model' (12.3 MB)

    Export complete (5.2s) Results saved to D:\python_YOLO_test Predict: yolo predict task=detect model=yolov8n_openvino_model imgsz=640
    Validate: yolo val task=detect model=yolov8n_openvino_model imgsz=640 data=coco.yaml
    Visualize: https://netron.app

    image 1/1 D:\python_YOLO_test\bus.jpg: 640x480 4 persons, 1 bus, 1 stop sign, 324.0ms Speed: 24.0ms preprocess, 324.0ms inference, 21.0ms postprocess per image at shape (1, 3, 640, 480) Loading yolov8n_openvino_model for OpenVINO inference... Using OpenVINO LATENCY mode for batch=1 inference...

    Process finished with exit code -1073741819 (0xC0000005)

    Process finished with exit code -1073740791 (0xC0000409) The model was successfully exported but did not perform inference, instead, it terminated directly. How should I resolve this issue?

    1. Glenn JocherGlenn JocherUltralytics(edited)

      @EduardoNew the exit code 0xC0000005 often indicates a memory access violation. Let's address this step-by-step:

      1. Path Format: Try using forward slashes in your OpenVINO model path: "yolov8n_openvino_model/" instead of backslashes.

      2. OpenVINO Version: Ultralytics requires openvino>=2024.0.0. Try upgrading to a recent release:

      pip install -U openvino
      1. Hardware Compatibility: Ensure you have the latest Intel drivers installed from Intel's official site.

      If issues persist, please create a Minimum Reproducible Example and open a GitHub Issue following our bug report guidelines. We'll be happy to investigate further!

  3. Ranjith3436Ranjith3436GitHub

    You mentioned that the core CPU inference model takes 23.53 ms. I would like to know if the processing time decreases when we increase the number of CPU cores ? currently i am using 12 thread CPU.

    1. Glenn JocherGlenn JocherUltralytics

      @Ranjith3436 increasing the number of CPU cores can potentially reduce processing time, as it allows for parallel processing, but the actual speedup will depend on the model's ability to leverage multi-threading effectively.

    3. Glenn JocherGlenn JocherUltralytics

      To assess a model's ability to utilize multiple CPU cores, you can check its support for multi-threading or parallel processing in the documentation or by running benchmarks on your hardware configuration.

  4. kimchi2020kimchi2020GitHub

    hey i am working on object detection in videos and came across openvino that helps in increasing speed. how can i use openvino and yolov8 together to find objects in a video

    1. Glenn JocherGlenn JocherUltralytics

      @kimchi2020 hi there! It's great to hear that you're exploring OpenVINO to enhance the speed of your object detection tasks in videos. OpenVINO can indeed provide significant performance improvements, especially on Intel hardware.

      To get started with using YOLOv8 and OpenVINO together, follow these steps:

      Exporting YOLOv8 to OpenVINO Format

      1. Export the Model: You can export your YOLOv8 model to the OpenVINO format using the following Python code:

        from ultralytics import YOLO

# Load a YOLOv8n PyTorch model
model = YOLO("yolov8n.pt")

# Export the model to OpenVINO format
model.export(format="openvino")  # creates 'yolov8n_openvino_model/'

        Alternatively, you can use the CLI:

        yolo export model=yolov8n.pt format=openvino  # creates 'yolov8n_openvino_model/'

      2. Run Inference with the Exported Model: After exporting, you can run inference using the exported OpenVINO model. Here’s how you can do it in Python:

        from ultralytics import YOLO

# Load the exported OpenVINO model
ov_model = YOLO("yolov8n_openvino_model/")

# Run inference
results = ov_model("https://ultralytics.com/images/bus.jpg")

        Or using the CLI:

        yolo predict model=yolov8n_openvino_model source='https://ultralytics.com/images/bus.jpg'
      Running Inference on Videos

      To process videos, you can modify the inference code to read frames from a video file and run the model on each frame. Here’s an example in Python:

      import cv2
from ultralytics import YOLO

# Load the exported OpenVINO model
model = YOLO("yolov8n_openvino_model/")

# Open the video file
video_path = "path/to/your/video.mp4"
cap = cv2.VideoCapture(video_path)

# Loop through the video frames
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Run inference on the frame
    results = model(frame)

    # Visualize the results on the frame
    annotated_frame = results[0].plot()

    # Display the annotated frame
    cv2.imshow("YOLOv8 OpenVINO Inference", annotated_frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object and close the display window
cap.release()
cv2.destroyAllWindows()
      Additional Resources

      For more detailed information and advanced usage, you can refer to the OpenVINO documentation. This guide covers everything from exporting models to running inference and optimizing performance.

      Feel free to reach out if you have any more questions. Happy coding! 🚀

    2. kimchi2020kimchi2020GitHub

      Traceback (most recent call last): File "/home/kimchi/Desktop/yolov8/onnx.py", line 18, in results = model(frame) ^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/model.py", line 174, in call return self.predict(source, stream, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/model.py", line 442, in predict return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/predictor.py", line 168, in call return list(self.stream_inference(source, model, *args, **kwargs)) # merge list of Result into one ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 35, in generator_context response = gen.send(None) ^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/predictor.py", line 234, in stream_inference self.model.warmup(imgsz=(1 if self.model.pt or self.model.triton else self.dataset.bs, 3, *self.imgsz)) File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/nn/autobackend.py", line 626, in warmup import torchvision # noqa (import here so torchvision import time not recorded in postprocess time) ^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/init.py", line 6, in from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/models/init.py", line 2, in from .convnext import * File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/models/convnext.py", line 8, in from ..ops.misc import Conv2dNormActivation, Permute File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/ops/init.py", line 1, in from ._register_onnx_ops import _register_custom_op File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torchvision/ops/_register_onnx_ops.py", line 5, in from torch.onnx import symbolic_opset11 as opset11 File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torch/onnx/init.py", line 59, in from ._internal.onnxruntime import ( File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torch/onnx/_internal/onnxruntime.py", line 36, in import onnx File "/home/kimchi/Desktop/yolov8/onnx.py", line 18, in results = model(frame) ^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/model.py", line 174, in call return self.predict(source, stream, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/model.py", line 442, in predict return self.predictor.predict_cli(source=source) if is_cli else self.predictor(source=source, stream=stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/predictor.py", line 168, in call return list(self.stream_inference(source, model, *args, **kwargs)) # merge list of Result into one ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 35, in generator_context response = gen.send(None) ^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/engine/predictor.py", line 261, in stream_inference self.results = self.postprocess(preds, im, im0s) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/models/yolo/detect/predict.py", line 25, in postprocess preds = ops.non_max_suppression( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kimchi/Desktop/yolov8/myvenv/lib/python3.12/site-packages/ultralytics/utils/ops.py", line 291, in non_max_suppression i = torchvision.ops.nms(boxes, scores, iou_thres) # NMS ^^^^^^^^^^^^^^^ AttributeError: partially initialized module 'torchvision' has no attribute 'ops' (most likely due to a circular import). Did you mean: 'os'?

      I am getting this error, how to get rid of it and what does it mean

    3. Glenn JocherGlenn JocherUltralytics

      Hi @kimchi2020,

      Thank you for reaching out! The error you're encountering seems to be related to a circular import issue with the torchvision library. This can happen if there's a naming conflict or an import loop in your code.

      Steps to Resolve the Issue:

      1. Check for Naming Conflicts: Ensure that your script name (onnx.py) is not conflicting with any library names. Rename your script to something unique, like run_inference.py.

      2. Verify Installation: Make sure you have the latest versions of torch, torchvision, and ultralytics installed. You can update them using:

        pip install --upgrade torch torchvision ultralytics

      3. Reproduce the Issue: If the issue persists, please provide a minimum reproducible example. This helps us diagnose the problem more effectively. You can refer to our guide on creating a reproducible example here: Minimum Reproducible Example.

      Example Code for Inference with OpenVINO:

      Here's a quick example of how to export your YOLOv8 model to OpenVINO and run inference on a video:

      import cv2
from ultralytics import YOLO

# Load a YOLOv8n PyTorch model
model = YOLO("yolov8n.pt")

# Export the model to OpenVINO format
model.export(format="openvino")  # creates 'yolov8n_openvino_model/'

# Load the exported OpenVINO model
ov_model = YOLO("yolov8n_openvino_model/")

# Open the video file
video_path = "path/to/your/video.mp4"
cap = cv2.VideoCapture(video_path)

# Loop through the video frames
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Run inference on the frame
    results = ov_model(frame)

    # Visualize the results on the frame
    annotated_frame = results[0].plot()

    # Display the annotated frame
    cv2.imshow("YOLOv8 OpenVINO Inference", annotated_frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object and close the display window
cap.release()
cv2.destroyAllWindows()

      For more detailed information on exporting and using OpenVINO, please refer to our OpenVINO Integration Guide.

      Feel free to reach out if you have any more questions or need further assistance. Happy coding! 🚀

    4. YA12SHYAMYA12SHYAMGitHub

      Hi @glenn-jocher , what if i want to use intel GPU's for the inference, using device=GPU doesn't seems to works , it looks for cuda i.e nvidia gpu

    5. Glenn JocherGlenn JocherUltralytics(edited)

      To use Intel GPUs for inference, export your YOLOv8 model to OpenVINO format with model.export(format="openvino"), then run inference with the exported model and select the GPU with device="intel:gpu", for example ov_model("bus.jpg", device="intel:gpu"). For more details, please refer to the OpenVINO integration guide.

    6. YA12SHYAMYA12SHYAMGitHub

      ok, just to confirm i can't use ultralytics python api for inference with intel gpu like i usually do within the ultralytics ecosystem for example ?

      from ultralytics import YOLO
model =  YOLO(path/to/model)

      here the path/to/model works with onnx, engine, opnevino;s xml , but it use cpu in case of openvino , i want to use my intel GPU

    7. Glenn JocherGlenn JocherUltralytics(edited)

      @YA12SHYAM to use Intel GPUs with YOLOv8, export your model to OpenVINO format and select the GPU at inference with device="intel:gpu", e.g. model("bus.jpg", device="intel:gpu"). For detailed steps, refer to the OpenVINO integration guide.

  5. kanishkaswkanishkaswGitHub

    Hi Im getting the following error when attempting to convert the model to openVino. Tried with both 2024.1.0 and 2024.0.0. Im on ubuntu 20.04 with python 3.8 $ yolo export model=runs/classify/train/weights/best.pt format=openvino Ultralytics YOLOv8.2.31 🚀 Python-3.8.10 torch-2.3.1+cu121 CPU (Intel Core(TM) i9-7900X 3.30GHz) YOLOv8n-cls summary (fused): 73 layers, 1438723 parameters, 0 gradients, 3.3 GFLOPs

    PyTorch: starting from 'runs/classify/train/weights/best.pt' with input shape (1, 3, 320, 320) BCHW and output shape(s) (1, 3) (2.8 MB) OpenVINO: export failure ❌ 0.0s: module 'openvino' has no attribute 'version' Traceback (most recent call last): File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/bin/yolo", line 8, in sys.exit(entrypoint()) File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/cfg/init.py", line 591, in entrypoint getattr(model, mode)(**overrides) # default args from model File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/model.py", line 602, in export return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model) File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/exporter.py", line 308, in call f[3], _ = self.export_openvino() File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/exporter.py", line 142, in outer_func raise e File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/exporter.py", line 137, in outer_func f, model = inner_func(*args, **kwargs) File "/home/Kanishka/v5.0/YOLOV8_classification/.venv/lib/python3.8/site-packages/ultralytics/engine/exporter.py", line 454, in export_openvino LOGGER.info(f"\n{prefix} starting export with openvino {ov.version}...") AttributeError: module 'openvino' has no attribute 'version'

    1. Paula DerrengerPaula DerrengerUltralytics(edited)

      @kanishkasw hi there,

      Thank you for reaching out and providing detailed information about the issue you're encountering. It looks like you're experiencing an error when attempting to export your model to OpenVINO format. Let's work through this together.

      First, let's ensure that you are using the latest versions of torch and ultralytics. Please run the following commands to update your packages:

      pip install --upgrade torch ultralytics

      Next, it seems the error is related to the OpenVINO package. The error message module 'openvino' has no attribute '__version__' suggests there might be an issue with the OpenVINO installation. Please make sure you have a recent version of OpenVINO installed (Ultralytics requires openvino>=2024.0.0). You can install it using the following command:

      pip install -U openvino

      Once you've updated the packages, try exporting the model again:

      yolo export model=runs/classify/train/weights/best.pt format=openvino

      If the issue persists, could you please provide a minimum reproducible example? This will help us investigate the problem more effectively. You can find guidance on creating a minimum reproducible example here: Minimum Reproducible Example.

      Additionally, here's a Python example to ensure everything is set up correctly:

      from ultralytics import YOLO

# Load a YOLOv8n PyTorch model
model = YOLO("runs/classify/train/weights/best.pt")

# Export the model to OpenVINO format
model.export(format="openvino")  # creates 'best_openvino_model/'

# Load the exported OpenVINO model
ov_model = YOLO("best_openvino_model/")

# Run inference
results = ov_model("https://ultralytics.com/images/bus.jpg")

      For more detailed instructions on using OpenVINO, please refer to our OpenVINO Integration Guide.

      Feel free to reach out if you have any further questions or run into any other issues. We're here to help! 😊

  6. renato-rodrigrenato-rodrigGitHub

    hello everything is fine? I have a question about the results exposed in the article above. Is the time shown in the graphs in relation to the inference time that the model takes to return the detections or is it the time for image reading, image pre-processing and model inference? I ask this because on my computer (intel core i7 12 generation CPU) the yolov8n model ran with an inference time of 40ms and the gain from using OpenVINO was not that great.

    1. Paula DerrengerPaula DerrengerUltralytics

      @renato-rodrig hello! 👋 Regarding your question about the times shown in the graphs, they typically refer to the total inference time, which includes the actual model inference as well as the time taken for pre-processing the input data (like image reading and resizing).

      The difference you observed with OpenVINO on your Intel Core i7 may vary due to several factors such as the specific configuration of your system, the type of tasks being run, and the optimizations that the YOLOv8n model can leverage on that platform. Generally, OpenVINO tends to show more significant performance gains on integrated Intel GPUs and some specific CPU configurations.

      If you haven't seen a major performance improvement, you might want to look into further tuning your OpenVINO deployment or explore using other optimization techniques provided by OpenVINO. Additionally, ensure all drivers and software components are up to date for optimal performance.

      Here’s a quick example of ensuring you're using the model in the most optimized manner:

      from ultralytics import YOLO

# Load YOLOv8n PyTorch model
model = YOLO('yolov8n.pt')

# Export the model to OpenVINO
model.export(format='openvino')  # Creates 'yolov8n_openvino_model/'

# Load the exported OpenVINO model
ov_model = YOLO('yolov8n_openvino_model/')

# Run inference to see if performance improves
results = ov_model('https://ultralytics.com/images/bus.jpg')

      Feel free to experiment with different configurations and settings! 😊

      1
  7. YouROS12YouROS12GitHub

    I wanna do inferences with my exported pre-trained yolov8 model using Raspberry PI with a Neural Network Computing 2 (NCS2) plugged in.. but its doesn't support opset 11 or above.. only opset10.. I specified onnx opset to 10 and convert it to Openvino but it didn't work.

    Anyway please to export openvino model directly in opset10 ? Does even yolov8 support this specified opset format ?

    Thank you.

    1. Glenn JocherGlenn JocherUltralytics(edited)

      @YouROS12 the current OpenVINO export no longer goes through ONNX: Ultralytics converts the PyTorch model directly to OpenVINO IR with openvino.convert_model, so the ONNX opset argument does not apply and the old mo Model Optimizer step isn't needed. Export with:

      from ultralytics import YOLO

# Load a pretrained YOLOv8 model
model = YOLO('yolov8n.pt')

# Export the model to OpenVINO
model.export(format='openvino')  # creates 'yolov8n_openvino_model/'

      The export requires openvino>=2024.0.0. If your target requires an older OpenVINO that cannot load these models, you can still try model.export(format='onnx', opset=10) and convert the ONNX file with that toolkit's own tools, but some YOLOv8 operations may not be supported in older opsets, which could lead to errors during export or inference.

    2. YouROS12YouROS12GitHub

      Where can I find documentation about YOLOv8's operations with opset 10 ? Updating OpenVINO toolkit is not an option because NCS2 works only at opset 10 and lower.

    3. Paula DerrengerPaula DerrengerUltralytics

      @YouROS12 the documentation for ONNX opset versions and the operations they support can be found on the ONNX GitHub repository or the official ONNX documentation. However, there isn't specific documentation for YOLOv8's operations with opset 10, as this would be a combination of YOLOv8's implementation details and ONNX's opset specifications.

      To check which operations are supported in opset 10, you can refer to the ONNX Operators documentation for that specific version. This documentation lists all the operators available in opset 10 and their details.

      For YOLOv8, you would need to look at the model's architecture and identify which operations are used. Then, cross-reference these operations with the ONNX opset 10 documentation to see if they are supported. If certain operations are not supported in opset 10, you might need to modify the model or the export script to use alternative operations that are compatible with opset 10.

      Since updating OpenVINO is not an option for you, if you find that YOLOv8 uses operations not supported in opset 10, you might need to consider using a different model architecture that is compatible with opset 10, or reach out to the community or Ultralytics for further assistance on how to proceed with your specific use case and hardware constraints.