开始使用

将 YOLO26 模型导出为 TFLite Edge TPU 格式

在计算能力有限的设备(例如移动设备或嵌入式系统)上部署计算机视觉模型可能颇具挑战。使用针对更快性能进行优化的模型格式可以简化这一过程。TensorFlow Lite Edge TPU(又称 TFLite Edge TPU)模型格式旨在以尽可能低的功耗,为神经网络提供高性能。

导出为 TFLite Edge TPU 格式,可让你优化 Ultralytics YOLO26 模型,实现高速、低功耗推理。本指南将带你了解如何将模型转换为 TFLite Edge TPU 格式,让模型更轻松地在各种移动和嵌入式设备上发挥出色性能。

为什么要导出为 TFLite Edge TPU?#

将模型导出到 TensorFlow Edge TPU,可让机器学习任务运行得更快、更高效。这项技术适用于功耗、计算资源和连接能力有限的应用。Edge TPU 是 Google 面向边缘设备推出的 Tensor Processing Unit (TPU) 硬件加速器。它能加速边缘设备上的 TensorFlow Lite 模型。下图展示了相关流程示例。

TensorFlow Lite Edge TPU compilation workflow

Edge TPU 支持量化模型。量化可以在几乎不损失精度的情况下缩小模型并提升速度,非常适合资源有限的边缘计算场景。量化可降低延迟并支持快速本地数据处理,使应用能够快速响应且不依赖云端。本地处理也能保护用户数据的隐私和安全,因为数据不会发送到远程服务器。

TFLite Edge TPU 的主要功能#

以下是让 TFLite Edge TPU 成为开发者优选模型格式的主要特性:

  • 针对边缘设备优化性能:TFLite Edge TPU 通过量化、模型优化、硬件加速和编译器优化,实现神经网络快速推理。其精简的架构也让它体积更小、更具成本效益。

  • 高计算吞吐量:TFLite Edge TPU 结合专用硬件加速和高效运行时执行,实现高计算吞吐量。它非常适合在边缘设备上部署性能要求严格的机器学习模型。

  • 高效矩阵计算:TensorFlow Edge TPU 针对矩阵运算进行了优化,而矩阵运算对于神经网络计算至关重要。这种高效性对于机器学习模型尤为重要,尤其是那些需要进行大量复杂矩阵乘法和变换的模型。

TFLite Edge TPU 的部署选项#

在将 YOLO26 模型导出为 TFLite Edge TPU 格式之前,先了解一下这类模型的典型应用场景会很有帮助。

TFLite Edge TPU 为机器学习模型提供多种部署选项,包括:

  • 设备端部署:TensorFlow Edge TPU 模型可以直接部署到移动设备和嵌入式设备上。设备端部署让模型能够直接在硬件上运行,无需连接云端;你可以将模型嵌入应用程序包中,也可以按需下载模型。

  • 混合部署:混合方式结合了设备端部署和云端部署,为机器学习模型的部署提供灵活且可扩展的解决方案。其优势包括通过设备端处理快速响应,以及利用云计算完成更复杂的计算。

支持的任务#

Edge TPU 导出支持 Ultralytics 七项任务中的六项。语义分割仅适用于 YOLO26,因为只有 YOLO26 系列包含该检测头。Edge TPU 不支持深度估计,因为其 INT8 模型会输出 Edge TPU 编译器无法解析的 EXP v2 运算符。

任务YOLOv8YOLO11YOLO26
检测✅✅✅
分割✅✅✅
语义分割❌❌✅
深度估计❌❌❌
分类✅✅✅
姿态✅✅✅
OBB✅✅✅

Edge TPU 编译器会将其支持的运算映射到加速器上,并将其余运算留给 CPU 执行,因此支持某项任务并不意味着该任务的每项运算都会在 TPU 上运行。

将 YOLO26 模型导出为 TFLite Edge TPU 格式#

将 YOLO26 模型转换为 TensorFlow Edge TPU 格式,可以扩展模型兼容性并提高部署灵活性。

安装#

要安装所需软件包,请运行:

安装
# Install the required package for YOLO26
pip install ultralytics

有关安装流程的详细说明和最佳实践,请查看我们的 Ultralytics 安装指南。安装 YOLO26 所需的软件包时,如果遇到困难,请查阅我们的常见问题指南,获取解决方案和建议。

用法#

所有 Ultralytics YOLO26 模型均支持开箱即用的导出,便于你将模型集成到首选的部署工作流中。你可以查看支持的导出格式和配置选项的完整列表,为你的应用选择最佳配置。

TFLite Edge TPU 格式支持导出、预测和验证模式。推理和验证在 Coral Edge TPU 硬件上运行。导出模型后,加载导出的模型即可运行推理或验证其准确率。

导出
from ultralytics import YOLO

# 加载 YOLO26 模型
model = YOLO("yolo26n.pt")

# 将模型导出为 TFLite Edge TPU 格式
model.export(format="edgetpu")  # 创建 'yolo26n_saved_model/yolo26n_full_integer_quant_edgetpu.tflite'
预测
from ultralytics import YOLO

# 加载导出的 TFLite Edge TPU 模型
model = YOLO("yolo26n_saved_model/yolo26n_full_integer_quant_edgetpu.tflite")

# 运行推理
results = model("https://ultralytics.com/images/bus.jpg")
验证
from ultralytics import YOLO

# 加载导出的 TFLite Edge TPU 模型
model = YOLO("yolo26n_saved_model/yolo26n_full_integer_quant_edgetpu.tflite")

# 在 COCO8 数据集上验证准确率
metrics = model.val(data="coco8.yaml")

导出参数#

参数类型默认值说明
formatstr'edgetpu'导出模型的目标格式,用于定义与各种部署环境的兼容性。
imgszint 或 tuple640模型输入所需的图像尺寸。正方形图像可使用整数;若要指定具体尺寸,则可使用元组 (height, width)。
quantizeint 或 str8/自动量化精度。Edge TPU 要求使用 8 (INT8),且会自动启用;它能压缩模型并加快边缘设备上的推理,同时将精度损失降至最低。此选项取代了已弃用的 half/int8 标志。
opsetintNone指定中间 ONNX 图的 ONNX opset 版本。如果未设置,则使用 opset 18(PyTorch < 2.4 时使用较低版本),且不高于已安装的 onnx 版本。
datastrNone数据集 YAML 文件路径,量化时必须提供;分类任务则接受数据集目录或内置数据集名称。如果使用 quantize=8 时未指定,Ultralytics 会根据模型任务选择默认校准数据集。
fractionfloat、int 或 list1.0校准子集,可按比例、图像数量或 [train, val, test] 比例/数量指定。包含两个项目的列表会让 test 保持完整,而 0 会跳过它。
devicestrNone指定导出所用的设备:CPU (device=cpu)。
提示

导出到 Edge TPU 时,请使用 x86 Linux 机器。

有关导出流程的更多详情,请访问 Ultralytics 导出文档页面。

部署导出的 YOLO26 TFLite Edge TPU 模型#

成功将 Ultralytics YOLO26 模型导出为 TFLite Edge TPU 格式后,就可以部署模型了。运行 TFLite Edge TPU 模型的首选且推荐的第一步,是使用 YOLO("model_edgetpu.tflite") 方法,具体如前面的用法代码片段所示。

若要详细了解如何部署 TFLite Edge TPU 模型,请参阅以下资源:

总结#

本指南介绍了如何将 Ultralytics YOLO26 模型导出为 TFLite Edge TPU 格式。按照上述步骤操作,你就能在边缘设备上以更快的速度和更低的功耗运行计算机视觉应用。

如需了解更多用法详情,请访问Coral Edge TPU 官方网站。

常见问题#

  • 要将 YOLO26 模型导出为 TFLite Edge TPU 格式,可以按照以下步骤操作:

    用法
    from ultralytics import YOLO
    
    # 加载 YOLO26 模型
    model = YOLO("yolo26n.pt")
    
    # 将模型导出为 TFLite Edge TPU 格式
    model.export(format="edgetpu")  # 创建 'yolo26n_saved_model/yolo26n_full_integer_quant_edgetpu.tflite'
    
    # 加载导出的 TFLite Edge TPU 模型
    edgetpu_model = YOLO("yolo26n_saved_model/yolo26n_full_integer_quant_edgetpu.tflite")
    
    # 运行推理
    results = edgetpu_model("https://ultralytics.com/images/bus.jpg")

    有关将模型导出为其他格式的完整详情,请参阅我们的导出指南。

  • 将 YOLO26 模型导出为 TFLite Edge TPU 格式有多项优势:

    • 性能优化:以极低的功耗实现高速神经网络推理。
    • 降低延迟:无需依赖云端,即可快速在本地处理数据。
    • 增强隐私保护:本地处理可确保用户数据的隐私和安全。

    因此,这种格式非常适合边缘计算应用,因为边缘设备的功耗和计算资源有限。

  • 可以,TensorFlow Lite Edge TPU 模型可以直接部署到移动设备和嵌入式设备上。这种部署方式让模型能够直接在硬件上运行,从而实现更快、更高效的推理。有关集成示例,请查看我们的在 Raspberry Pi 上部署 Coral Edge TPU 指南。

  • TFLite Edge TPU 模型的常见用例包括:

    • 智能摄像头:增强实时图像和视频分析能力。
    • IoT 设备:实现智能家居和工业自动化。
    • 医疗保健:加速医学影像处理和诊断。
    • 零售:改善库存管理和客户行为分析。

    这些应用受益于 TFLite Edge TPU 模型的高性能和低功耗。了解更多使用场景。

  • 如果在导出或部署 TFLite Edge TPU 模型时遇到问题,请参阅我们的常见问题指南获取故障排查建议。该指南介绍了常见问题及其解决方案,帮助你确保模型顺利运行。如需更多支持,请访问我们的帮助中心。

评论30

提出问题、分享结果,或帮助其他人。

  1. faizaanscnzGitHub

    I am running a float32 TFLite model on Pixel 8a using the GPU. How can I run an Edge TPU model on the hardware to get faster FPS? I’m currently getting 8 FPS, but I need 20+. (Kivy app)

  2. walissongGitHub

    I would like to know if this tutorial works with yolov8n. I've been trying for many days and I'm not able to convert it to edgetpu.tflite.

  3. dbroGitHub

    Hello. This seems to be broken for yolo11, although it works for yolov8. After running these two commands in Google colab, the output file is less than 1kb and it generates an error.

    ! pip install ultralytics ! yolo mode=export model=yolo11n.pt format=edgetpu imgsz=320

    The final step to run the EdgeTPU compiler seems to be where the problem is:

    ... TensorFlow SavedModel: export success ✅ 112.9s, saved as 'yolo11n_saved_model' (33.8 MB)

    Edge TPU: starting export with Edge TPU compiler 16.0.384591198... Edge TPU: running 'edgetpu_compiler --out_dir "yolo11n_saved_model" --show_operations --search_delegate --delegate_search_step 30 --timeout_sec 180 "yolo11n_saved_model/yolo11n_full_integer_quant.tflite"' Edge TPU Compiler version 16.0.384591198 Searching for valid delegate with step 30 Try to compile segment with 677 ops ERROR: Didn't find op for builtin opcode 'CONV_2D' version '6'. An older version of this builtin might be supported. Are you using an old TFLite binary with a newer model?

    ERROR: Registration failed.

    Invalid model: yolo11n_saved_model/yolo11n_full_integer_quant.tflite Model could not be parsed Edge TPU: export success ✅ 0.0s, saved as 'yolo11n_saved_model/yolo11n_full_integer_quant_edgetpu.tflite' (0.0 MB) ...

  4. manu13008GitHub

    Hi, Once I exported my best.pt model into edgetpu tflite, I tried to run an inference on an image as the code above but I got the following error :

    OSError: libedgetpu.so.1: cannot open shared object file: No such file or directory

    Do you have any insights of how resolving this ?

    Thanks :)

    1. Glenn JocherUltralytics

      @manu13008 it seems like the error is due to the missing Edge TPU runtime library. Please ensure you have installed the Edge TPU runtime on your device. You can find installation instructions in the Edge TPU documentation. If you need further assistance, feel free to check the resources there.

  5. Abdullah-OryGitHub

    Hi friends, i am using google colap to compile yolo11n.pt into edgetpu format, since i have orange pi 5 pro which is arm, so when i am trying to do it in my laptop i got this Ultralytics 8.3.9 🚀 Python-3.10.12 torch-2.4.1+cu121 CPU (Intel Xeon 2.20GHz) YOLO11n summary (fused): 238 layers, 2,616,248 parameters, 0 gradients, 6.5 GFLOPs

    PyTorch: starting from '/content/yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB)

    TensorFlow SavedModel: starting export with tensorflow 2.17.0...

    ONNX: starting export with onnx 1.17.0 opset 19... ONNX: slimming with onnxslim 0.1.34... ONNX: export success ✅ 3.0s, saved as '/content/yolo11n.onnx' (10.3 MB) TensorFlow SavedModel: starting TFLite export with onnx2tf 1.22.3...

    Automatic generation of each OP name started ======================================== Automatic generation of each OP name complete!

    Model loaded ========================================================================

    Model conversion started ============================================================ saved_model output started ========================================================== saved_model output complete! Float32 tflite output complete! Float16 tflite output complete! Dynamic Range Quantization tflite output complete! Input signature information for quantization signature_name: serving_default input_name.0: images shape: (1, 640, 640, 3) dtype: <dtype: 'float32'> INT8 Quantization tflite output complete! Full INT8 Quantization tflite output complete! Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/onnx2tf/onnx2tf.py", line 1566, in convert tflite_model = converter.convert() File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1231, in wrapper return self._convert_and_export_metrics(convert_func, *args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1183, in _convert_and_export_metrics result = convert_func(self, *args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1562, in convert return self._convert_from_saved_model(graph_def) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1424, in _convert_from_saved_model return self._optimize_tflite_model( File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/convert_phase.py", line 215, in wrapper raise error from None # Re-throws the exception. File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/convert_phase.py", line 205, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1127, in _optimize_tflite_model model = self._quantize( File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 767, in _quantize return calibrate_quantize.calibrate_and_quantize( File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/convert_phase.py", line 215, in wrapper raise error from None # Re-throws the exception. File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/convert_phase.py", line 205, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/optimize/calibrator.py", line 194, in calibrate_and_quantize return self._calibrator.QuantizeModel( RuntimeError: Quantized dimension for tensor property and quantization parameters do not match. Got 3 and 0 respectively.

    WARNING: INT8 Quantization with int16 activations tflite output failed. Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/onnx2tf/onnx2tf.py", line 1597, in convert tflite_model = converter.convert() File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1231, in wrapper return self._convert_and_export_metrics(convert_func, *args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1183, in _convert_and_export_metrics result = convert_func(self, *args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1562, in convert return self._convert_from_saved_model(graph_def) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1424, in _convert_from_saved_model return self._optimize_tflite_model( File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/convert_phase.py", line 215, in wrapper raise error from None # Re-throws the exception. File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/convert_phase.py", line 205, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 1127, in _optimize_tflite_model model = self._quantize( File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/lite.py", line 767, in _quantize return calibrate_quantize.calibrate_and_quantize( File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/convert_phase.py", line 215, in wrapper raise error from None # Re-throws the exception. File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/convert_phase.py", line 205, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/optimize/calibrator.py", line 194, in calibrate_and_quantize return self._calibrator.QuantizeModel( RuntimeError: Quantized dimension for tensor property and quantization parameters do not match. Got 3 and 0 respectively.

    WARNING: Full INT8 Quantization with int16 activations tflite output failed. TensorFlow SavedModel: export success ✅ 248.1s, saved as '/content/yolo11n_saved_model' (36.1 MB) Edge TPU: WARNING ⚠️ Edge TPU known bug https://github.com/ultralytics/ultralytics/issues/1185

    Edge TPU: starting export with Edge TPU compiler 16.0.384591198... Edge TPU: running 'edgetpu_compiler -s -d -k 10 --out_dir "/content/yolo11n_saved_model" "/content/yolo11n_saved_model/yolo11n_full_integer_quant.tflite"' Edge TPU: export success ✅ 3.9s, saved as '/content/yolo11n_saved_model/yolo11n_full_integer_quant_edgetpu.tflite' (3.5 MB)

    Export complete (252.6s) Results saved to /content Predict: yolo predict task=detect model=/content/yolo11n_saved_model/yolo11n_full_integer_quant_edgetpu.tflite imgsz=640 int8 Validate: yolo val task=detect model=/content/yolo11n_saved_model/yolo11n_full_integer_quant_edgetpu.tflite imgsz=640 data=/usr/src/ultralytics/ultralytics/cfg/datasets/coco.yaml int8 Visualize: https://netron.app /content/yolo11n_saved_model/yolo11n_full_integer_quant_edgetpu.tflite

    1. Glenn JocherUltralytics

      @Abdullah-Ory it seems you're encountering an issue with INT8 quantization during the TFLite Edge TPU export. Please ensure your model is compatible with Edge TPU requirements, and consider checking the Ultralytics documentation for detailed guidance on exporting models to TFLite Edge TPU format. If the issue persists, reviewing the quantization parameters might help.

  6. MECHAAMERGitHub

    Hi,

    I trained a YOLOv8l model with VisDrone dataset, and attempted to convert the best weights.pt to Edge TPU TFlite format using:

    model = YOLO("best.pt") model.export(format="edgetpu")

    my board is Coral dev board.

    The conversion completed successfully, but the results indicate that 164 of operations will run on the CPU instead of the Edge TPU. only 223 operation will run using Edge TPU This CPU usage will likely cause performance delays.

    I'm wondering if there are any strategies or techniques to improve Edge TPU utilization and minimize CPU usage during inference for this model. Any suggestions or insights would be greatly appreciated.

    Thanks,

    1. Glenn JocherUltralytics

      @MECHAAMER to improve Edge TPU utilization, ensure your model is fully quantized and compatible with Edge TPU operations. Consider simplifying the model architecture or using supported operations. For detailed guidance, please refer to the Ultralytics Edge TPU documentation.

    2. MECHAAMERGitHub

      Hi, thank you for your support, I follow the attached link, and apply it using: model.export(format="edgetpu") , and I get this output:

      Number of operations that will run on Edge TPU: 133 Number of operations that will run on CPU: 364

      more details in the attached photo.

      I think with this big percent of ops that runs on the CPU, the model will be very slow and I will suffer from large latency.

      Do you have any suggestions?

    3. MECHAAMERGitHub

      Hi there,

      I'm following the Ultralytics Edge TPU documentation to train a YOLO model for object detection. However, I'm encountering suboptimal results.

      I suspect that the issue might lie in the model's compatibility with the TPU. It's possible that certain operations within the model are not optimized for the TPU, leading to delays.

      To address this, I'm considering using a YOLO model that is specifically designed to run efficiently on the Edge TPU. Do you have any recommendations for such models?

    4. Glenn JocherUltralytics

      For optimal Edge TPU performance, consider using a smaller YOLO model like YOLOv8n, which is more compatible with Edge TPU operations. Simplifying the model architecture can also help.

    5. MECHAAMERGitHub

      I'm encountering a "Segmentation fault" error while trying to run a segmented YOLOv5s model on my Coral Dev Board with an external Coral ML Accelerator (USB). I followed the recommended steps to export, compile, and segment the model using edgetpu_compiler.

      I follow these steps:

      I trained Yolov5su model, then I exported the model as tflite - 8int by using:
      model.export()
      
      then I use the edgetpu_compiler to compile the tflite model and segmented it into 2 models:
      edgetpu_compiler model.tflite --num_segments=2
      
      then I get 12 files, 8 models and 4 logs, the models are:
      1- best_float32_segment_0_of_2.tflite
      2- best_float32_segment_1_of_2.tflite
      3- best_float32_segment_0_of_2_edgetpu.tflite
      4- best_float32_segment_1_of_2_edgetpu.tflite
      5- best_full_integer_quant_segment_0_of_2.tflite
      6- best_full_integer_quant_segment_1_of_2.tflite
      7- best_full_integer_quant_segment_0_of_2_edgetpu.tflite
      8- best_full_integer_quant_segment_1_of_2_edgetpu.tflite

      I transfer the "best_full_integer_quant_segment_0_of_2_edgetpu.tflite" and "best_full_integer_quant_segment_1_of_2_edgetpu.tflite" to the coral board and I tried to run the models to detect object in a video, the code:

      
      import argparse
      import re
      import threading
      import time
      
      import numpy as np
      import cv2
      
      from pycoral.adapters import classify
      from pycoral.adapters import common
      import pycoral.pipeline.pipelined_model_runner as pipeline
      from pycoral.utils.dataset import read_label_file
      from pycoral.utils.edgetpu import list_edge_tpus
      from pycoral.utils.edgetpu import make_interpreter
      
      
      def _get_devices(num_devices):
        edge_tpus = list_edge_tpus()
      
        if len(edge_tpus) < num_devices:
          raise RuntimeError(
              'Not enough Edge TPUs detected, expected %d, detected %d.' %
              (num_devices, len(edge_tpus)))
      
        num_pci_devices = sum(1 for device in edge_tpus if device['type'] == 'pci')
      
        return ['pci:%d' % i for i in range(min(num_devices, num_pci_devices))] + [
            'usb:%d' % i for i in range(max(0, num_devices - num_pci_devices))
        ]
      
      
      def _make_runner(model_paths, devices):
        """Constructs PipelinedModelRunner given model paths and devices."""
        print('Using devices: ', devices)
        print('Using models: ', model_paths)
         
        print("  hey runner   1")
        if len(model_paths) != len(devices):
          raise ValueError('# of devices and #of model_paths should match')
        
        print("  hey runner   2", model_paths[0], devices[1])
       
        interpreters = []
        interpreter = make_interpreter(model_paths[0], devices[0])
        print("  hey runner   3")
        interpreter.allocate_tensors()
        print("  hey runner   4")
        interpreters.append(interpreter)
        print("  hey runner   5", model_paths[1], devices[0])
        
        interpreter2 = make_interpreter(model_paths[1], devices[1])
        print("  hey runner   6")
        interpreter.allocate_tensors()
        interpreter2.allocate_tensors()
        print("  hey runner   7")
        interpreters.append(interpreter2)
        print("  hey runner   8")
      
        return pipeline.PipelinedModelRunner(interpreters)
      
      
      def main():
        parser = argparse.ArgumentParser(
            formatter_class=argparse.ArgumentDefaultsHelpFormatter)
        parser.add_argument(
            '-m',
            '--models',
            required=True,
            help=('File path template of .tflite model segments, e.g.,'
                  'inception_v3_299_quant_segment_%d_of_2_edgetpu.tflite'))
        parser.add_argument(
            '-i', '--input', required=True, help='Image to be classified.')
        parser.add_argument('-l', '--labels', help='File path of labels file.')
        parser.add_argument(
            '-k',
            '--top_k',
            type=int,
            default=1,
            help='Max number of classification results')
        parser.add_argument(
            '-t',
            '--threshold',
            type=float,
            default=0.0,
            help='Classification score threshold')
        parser.add_argument(
            '-c',
            '--count',
            type=int,
            default=1,
            help='Number of times to run inference')
        args = parser.parse_args()
        labels = {0:"pedestrian", 1:"people", 2:"bicycle", 3:"car", 4:"van", 5:"truck", 6:"tricycle", 7:"awning-tricycle", 8:"bus", 9:"motor"}
        print("hey lets begin   1")
        result = re.search(r'^.*_segment_%d_of_(?P<num_segments>[0-9]+)_.*.tflite',
                           args.models)
        print("hey lets begin   2")
      
        if not result:
          print("hey lets begin    3")
      
          raise ValueError(
              '--models should follow *_segment%d_of_[num_segments]_*.tflite pattern')
        print("hey lets begin    4")
      
        num_segments = int(result.group('num_segments'))
        model_paths = [args.models % i for i in range(num_segments)]
        print("hey lets begin    5")
      
        devices = _get_devices(num_segments)
      
        print("hey lets begin    6")
      
        runner = _make_runner(model_paths, devices)
        
        print("hey lets begin      7")
      
      
        # Get the input name and size
        size = common.input_size(runner.interpreters()[0])
        
        print("hey lets begin     8")
      
        name = common.input_details(runner.interpreters()[0], 'name')
       
        print("hey lets begin    9")
      
        cap = cv2.VideoCapture(args.input)
        
        print("hey lets begin    10")
      
        if not cap.isOpened():
            print("Error opening video capture device")
            return
        
        print("hey lets begin    11")
      
        while True:
            ret, frame = cap.read()
            if not ret:
                break
      
            # Preprocess the frame
            height, width, _ = frame.shape
            new_height = int(640 * height / width)
            image = cv2.resize(frame, (640, new_height))
      
            # Run inference
            runner.push({name: image})
            
            output_details = runner.interpreters()[-1].get_output_details()[0]
            scale, zero_point = output_details['quantization']
        
            result = runner.pop()
      
            if result:
                # Process the results
                values, = result.values()
                scores = scale * (values[0].astype(np.int64) - zero_point)
                classes, scores = classify.get_classes_from_scores(result[0], args.top_k, args.threshold)
                
                for klass in classes:
                    print('%s: %.5f' % (labels.get(klass.id, klass.id), klass.score))
                
                print('-------RESULTS--------')
            else:
                print("No results")
      
      if __name__ == '__main__':
        main()
      then I try execute the python file:

      python3 test2tpu.py -m best_full_integer_quant_segment_%d_of_2_edgetpu.tflite -i "/coral/720p/Busy Road.mp4"

      the output that I get:

      hey lets begin 1 hey lets begin 2 hey lets begin 4 hey lets begin 5 hey lets begin 6 Using devices: ['pci:0', 'usb:0'] Using models: ['best_full_integer_quant_segment_0_of_2_edgetpu.tflite', 'best_full_integer_quant_segment_1_of_2_edgetpu.tflite'] hey runner 1 hey runner 2 best_full_integer_quant_segment_0_of_2_edgetpu.tflite usb:0 hey runner 3 hey runner 4 hey runner 5 best_full_integer_quant_segment_1_of_2_edgetpu.tflite pci:0 Segmentation fault

      Note: pycoral version: 2.0.0

    6. Glenn JocherUltralytics

      It seems like the segmentation fault might be due to compatibility issues with the model operations on the Edge TPU. Consider simplifying the model architecture or ensuring all operations are supported by the Edge TPU. You can also check the Common Issues guide for additional troubleshooting tips.

    7. MECHAAMERGitHub

      Can you help me to know more about: simplifying the model architecture or ensuring all operations are supported by the Edge TPU.

      I don't have access now to the records because I am outside office, as I remember In yolov5su, there are more than 200 process depends in TPU and only 25 process use CPU, Also the model working well on 1 TPU (with no segmentation).

    8. Glenn JocherUltralytics

      @MECHAAMER to improve Edge TPU compatibility, consider using a smaller model like YOLOv8n, which is more likely to use supported operations, or simplify the architecture by reducing layers and operations that are not TPU-optimized.

  7. Taufiq-AhmedGitHub

    Hi, I have train my custom dataset for detect fire and smoke with yolov8 nano. it's best model size is around 6MB and after exporting it using "yolo export model=yolov8n.pt format=edgetpu". it's size reduced to 3MB. But have to integrate this model in XIAO RP2040 MCU. So I need more light weight model for my edge device. In this case what I should? Please give me a solution.

    1. Glenn JocherUltralytics

      @Taufiq-Ahmed hi! It's great to hear that you've successfully trained your custom dataset with YOLOv8 nano. For further reducing the model size for your XIAO RP2040 MCU, you might want to explore additional quantization techniques. The TFLite Edge TPU format already applies full integer quantization, but you can experiment with pruning or other model compression techniques to further reduce the size. If you haven't already, ensure you're using the latest version of the Ultralytics package for the best optimization features. For more detailed guidance, you can check out our Edge TPU integration guide. Good luck with your deployment!

  8. samphonicGitHub

    So I have a tflite model and object detection is working, but I can't seem to figure out how to tell if (and how effectively) the model is actually using the Edge TPU hardware. Is there some way to confirm that the hardware is actually being used? Can I manually enable/disable it to compare inference time?

    1. Glenn JocherUltralytics

      @Sammy1Am to confirm if your model is utilizing the Edge TPU hardware, you can check the inference time and compare it with and without the Edge TPU. Typically, when the Edge TPU is in use, you should see a significant reduction in inference time. You can manually enable or disable the Edge TPU by using the edgetpu interpreter for the Edge TPU and the standard TFLite interpreter otherwise. For more details on setting this up, you might find the Edge TPU documentation helpful. If you need further assistance, feel free to explore our Ultralytics documentation.

    1. Glenn JocherUltralytics

      @FearL0rd thank you for your suggestion! Streamlining segmentation during export is indeed a valuable enhancement. For detailed guidance on exporting YOLOv8 models to TFLite Edge TPU format, you can refer to our documentation. This will help you optimize your models for high-speed, low-power inferencing on devices with limited computational power.

      For more information on using YOLO with a Dual TPU card, you can leverage the APEX_0 and APEX_1 devices to distribute the workload effectively. This setup can significantly boost performance and efficiency in your machine learning tasks.

      Feel free to explore the provided resources and let us know if you have any further questions! 🚀

    2. Glenn JocherUltralytics

      Hi @FearL0rd,

      For documentation on segmentation using APEX_0 and APEX_1, you can refer to the guide on exporting YOLOv8 models to TFLite Edge TPU format. This guide covers optimizing your models for high-speed, low-power inferencing on devices like the Coral Edge TPU.

      To utilize both APEX_0 and APEX_1 devices, you can distribute the workload effectively across the dual TPU setup, enhancing performance and efficiency. For more detailed instructions, please visit the relevant sections in our documentation.

      Feel free to reach out if you have any more questions! 😊