배포를 위한 TFLite 모델 내보내기(지원 중단됨)

지원 중단됨 — LiteRT로 대체됨

Ultralytics 8.4.83부터 독립형 tflite 내보내기 형식은 더 이상 사용되지 않으며 통합 Google LiteRT 형식으로 대체되었습니다. LiteRT(Lite Runtime)는 TensorFlow Lite의 차세대 기술이자 새 이름이며, 동일한 .tflite 모델을 내보냅니다. 이제 하나의 형식으로 모바일, 임베디드, 엣지, 브라우저 배포를 지원합니다.

format="tflite"은 계속 작동하지만 사용 중단 경고를 표시하고 대신 LiteRT 모델을 내보냅니다. 앞으로는 format="litert"을 사용하세요. 최신 내보내기 지침과 옵션은 LiteRT 내보내기 가이드를 참조하세요.

TensorFlow Lite edge deployment framework

엣지 또는 임베디드 기기에 컴퓨터 비전 모델을 배포하려면 온디바이스 추론에 최적화된 형식이 필요합니다.

기존 TensorFlow Lite 또는 TFLite 내보내기 형식은 엣지 애플리케이션에서 객체 탐지 및 이미지 분류와 같은 작업에 사용하도록 Ultralytics YOLO26 모델을 최적화했습니다. 이 가이드는 기존 TFLite 배포 맥락을 다룹니다. 새로 내보낼 때는 LiteRT를 사용하세요.

내보내기에 TFLite를 사용한 이유는 무엇인가요?#

2017년 5월 Google이 TensorFlow 프레임워크의 일부로 출시한 TensorFlow Lite, 즉 TFLite는 엣지 컴퓨팅이라고도 하는 온디바이스 추론을 위해 설계된 오픈 소스 딥러닝 프레임워크입니다. 개발자는 이 프레임워크의 도구를 사용해 학습된 모델을 모바일, 임베디드, IoT 디바이스뿐 아니라 일반 컴퓨터에서도 실행할 수 있었습니다.

TensorFlow Lite는 임베디드 Linux, Android, iOS 및 마이크로컨트롤러(MCU)를 비롯한 다양한 플랫폼을 지원했습니다. TFLite로 내보낸 모델을 사용하면 애플리케이션에서 모델을 로컬로 오프라인 실행할 수 있었습니다.

TFLite 모델의 주요 기능#

TFLite 모델은 개발자가 모바일, 임베디드 및 엣지 디바이스에서 모델을 실행할 수 있도록 지원하여 온디바이스 머신러닝을 구현하는 다양한 주요 기능을 제공합니다.

  • 온디바이스 최적화: TFLite는 온디바이스 ML에 최적화되어 있습니다. 데이터를 로컬에서 처리해 지연 시간을 줄이고, 개인정보를 전송하지 않아 프라이버시를 강화하며, 모델 크기를 최소화해 공간을 절약합니다.

  • 다양한 플랫폼 지원: TFLite는 Android, iOS, 임베디드 Linux 및 마이크로컨트롤러를 지원하는 폭넓은 플랫폼 호환성을 제공합니다.

  • 다양한 언어 지원: TFLite는 Java, Swift, Objective-C, C++ 및 Python을 비롯한 여러 프로그래밍 언어와 호환됩니다.

  • 높은 성능: 하드웨어 가속과 모델 최적화를 활용해 추론 속도를 높입니다.

측정된 성능(과거 기록)#

마이그레이션 전후 기록 동결

이 결과는 Android 16/API 36에서 12 GB LPDDR5X 메모리가 탑재된 Xiaomi 17을 사용해 Ultralytics Flutter 플러그인 0.6.8 및 LiteRT 2.1.5로 기록했습니다. 3 nm Snapdragon 8 Elite Gen 5(SM8850)에는 8코어 Qualcomm Oryon CPU(최대 4.6 GHz의 Prime 코어 2개와 최대 3.62 GHz의 Performance 코어 6개), Adreno GPU 및 Hexagon NPU가 탑재되어 있습니다. 이 표에서는 마이그레이션 중에 평가한 기존 onnx2tf INT8 TFLite 에셋과 후보 litert-torch 내보내기를 비교합니다. 이후 릴리스 에셋을 표준화된 내보내기 결과로 덮어썼으므로, 이 수치는 현재 v0.7.0 에셋의 바이트를 나타내지 않습니다. 최신 측정값은 LiteRT 성능 표를 참조하세요.

두 형식 모두 표시된 입력 크기로 동일한 연속 GPU 테스트를 수행했습니다. 각 셀에는 전체 시간(전처리 + 추론 + 후처리)과 그 아래 단계별 시간이 표시되어 있습니다. 네이티브 로그에 따르면 두 형식 모두 전체 그래프를 Adreno GPU의 LiteRT OpenCL(LITERT_CL)에 위임했습니다.

모델작업크기
(픽셀)
기존
onnx2tf INT8 TFLite
(ms)
후보
w8a32 LiteRT
(ms)
YOLO26n감지64014.0
1.8 / 8.1 / 4.2
13.5
1.9 / 8.1 / 3.5
YOLO26n-seg분할64030.1
1.9 / 20.3 / 8.0
28.6
1.8 / 20.1 / 6.7
YOLO26n-sem의미론적64026.4
1.9 / 16.4 / 8.1
32.9
1.8 / 23.0 / 8.2
YOLO26n-cls분류2243.5
0.9 / 2.2 / 0.4
3.2
1.0 / 2.2 / 0.1
YOLO26n-pose포즈64017.4
2.4 / 9.9 / 5.1
14.0
1.9 / 9.3 / 2.8
YOLO26n-obbOBB64013.9
3.0 / 8.3 / 2.7
13.0
2.9 / 7.9 / 2.3

동일한 마이그레이션 실행에서 YOLO26n 탐지 양자화 형식도 비교했습니다. 형식에 따라 달라지는 비교 가능 지표는 추론 시간입니다.

Android 형식GPU 추론(ms)GPU 컴파일
onnx2tf INT8(기존 TFLite)8.6예
LiteRT w8a32(후보)8.4예
LiteRT INT8(quantize=8)11.0예
LiteRT FP328.8예
LiteRT w8a16(quantize="w8a16")CPU 대체 실행아니요

이번 실행에서 w8a16은 CPU로 대체 실행되어 총 소요 시간이 약 660 ms였으며, GPU 형식은 약 17 ms였습니다. 이러한 결과를 바탕으로 w8a32을 출시했지만, 위임 호환성과 성능은 디바이스 및 런타임 버전에 따라 달라집니다. 대상 디바이스에서 벤치마크를 실행하고 런타임 로그에서 가속기 할당을 확인하세요.

TFLite의 배포 옵션#

LiteRT 대체 내보내기 예제를 살펴보기 전에, 일반적으로 TFLite 모델을 사용하는 방법을 알아보겠습니다.

TFLite는 머신러닝 모델을 위한 다음과 같은 다양한 온디바이스 배포 옵션을 제공합니다.

  • Android 및 iOS에 배포: TFLite를 사용하는 Android 및 iOS 애플리케이션은 엣지 기반 카메라 피드와 센서를 분석하여 객체를 탐지하고 식별할 수 있습니다. TFLite는 Swift 및 Objective-C로 작성된 네이티브 iOS 라이브러리도 제공합니다. 아래 아키텍처 다이어그램은 TensorFlow Lite를 사용해 학습된 모델을 Android 및 iOS 플랫폼에 배포하는 과정을 보여줍니다.

TensorFlow Lite deployment architecture for mobile

  • 임베디드 Linux 구현: Raspberry Pi에서 추론을 실행할 때 사용 사례의 속도 요구사항을 충족하지 못한다면, 내보낸 TFLite 모델을 사용해 추론을 가속할 수 있습니다. 또한 Coral Edge TPU 장치를 사용해 성능을 더 높일 수도 있습니다.

  • 마이크로컨트롤러에 배포: TFLite 모델은 메모리가 몇 KB에 불과한 마이크로컨트롤러 및 기타 디바이스에도 배포할 수 있습니다. 핵심 런타임은 Arm Cortex M3에서 16 KB만 사용하며 여러 기본 모델을 실행할 수 있습니다. 운영 체제 지원, 표준 C 또는 C++ 라이브러리, 동적 메모리 할당이 필요하지 않습니다.

TFLite 내보내기를 LiteRT로 대체하기#

새로운 내보내기에는 모델을 LiteRT로 변환하세요. 결과 모델은 .tflite 파일 확장자를 유지합니다.

설치#

필수 패키지를 설치하려면 다음을 실행하세요:

설치
# Install the required package for YOLO26
pip install ultralytics

설치 절차에 대한 자세한 지침과 모범 사례는 Ultralytics 설치 가이드를 참조하세요. YOLO26에 필요한 패키지를 설치하는 중 문제가 발생하면 해결 방법과 팁을 확인할 수 있는 일반적인 문제 가이드를 참조하세요.

사용법#

모든 Ultralytics YOLO26 모델은 별도의 설정 없이 내보내기를 지원하도록 설계되어 있어 원하는 배포 워크플로에 쉽게 통합할 수 있습니다. 애플리케이션에 가장 적합한 설정을 선택하려면 지원되는 내보내기 형식 및 구성 옵션 전체 목록을 확인하세요.

대체된 LiteRT 형식은 내보내기, 예측 및 검증 모드를 지원합니다. 모델을 내보낸 다음, 내보낸 .tflite 모델을 로드하여 추론을 실행하거나 정확도를 검증하세요.

내보내기
from ultralytics import YOLO

# YOLO26 모델 로드
model = YOLO("yolo26n.pt")

# 모델을 LiteRT 형식으로 내보내기
model.export(format="litert", imgsz=640)  # 분류에는 imgsz=224 사용
예측
from ultralytics import YOLO

# 내보낸 TFLite 모델 로드
model = YOLO("yolo26n.tflite")

# 추론 실행
results = model("https://ultralytics.com/images/bus.jpg")
검증
from ultralytics import YOLO

# 내보낸 TFLite 모델 로드
model = YOLO("yolo26n.tflite")

# COCO8 데이터셋에서 정확도 검증
metrics = model.val(data="coco8.yaml")

내보내기 인수#

LiteRT 내보내기 인수를 참조하세요. 내보내기 프로세스에 관한 자세한 내용은 내보내기에 관한 Ultralytics 문서 페이지를 참조하세요.

내보낸 YOLO26 TFLite 모델 배포하기#

Ultralytics YOLO26 모델을 LiteRT 형식으로 내보낸 뒤에는 결과물인 .tflite 모델을 배포할 수 있습니다. 앞서 사용 예제 코드에 설명된 것처럼, TFLite 모델을 실행할 때 가장 먼저 권장하는 방법은 YOLO("model.tflite") 메서드를 사용하는 것입니다. 다양한 다른 환경에서 TFLite 모델을 배포하는 자세한 지침은 다음 자료를 참조하세요.

  • Android: TensorFlow Lite를 Android 애플리케이션에 통합하는 빠른 시작 가이드로, 머신러닝 모델을 설정하고 실행하는 단계를 쉽게 따라 할 수 있도록 안내합니다.

  • iOS: TensorFlow Lite 모델을 iOS 애플리케이션에 통합하고 배포하는 개발자 가이드로, 단계별 지침과 자료를 제공합니다.

  • 엔드투엔드 예제: 이 페이지에서는 다양한 TensorFlow Lite 예제를 개괄하고, 개발자가 모바일 및 엣지 디바이스의 머신러닝 프로젝트에 TensorFlow Lite를 구현하는 데 도움이 되는 실용적인 애플리케이션과 튜토리얼을 소개합니다.

요약#

이 가이드에서는 기존 TFLite 배포 워크플로를 다룹니다. 새로운 내보내기에는 LiteRT를 사용해 엣지 컴퓨팅 환경용 .tflite 모델을 생성하세요.

사용법에 관한 자세한 내용은 TFLite 공식 문서를 참조하세요.

자주 묻는 질문#

  • 새 모델을 내보낼 때는 LiteRT 형식을 사용하세요. 먼저 다음 명령으로 필요한 패키지를 설치합니다.

    pip install ultralytics

    그런 다음 다음 코드 스니펫을 사용해 모델을 내보냅니다.

    from ultralytics import YOLO
    
    # YOLO26 모델 로드
    model = YOLO("yolo26n.pt")
    
    # 모델을 LiteRT 형식으로 내보내기
    model.export(format="litert", imgsz=640)  # 분류에는 imgsz=224 사용

    CLI 사용자는 다음 명령으로 내보낼 수 있습니다.

    yolo export model=yolo26n.pt format=litert imgsz=640 # use imgsz=224 for classification

    자세한 내용은 Ultralytics 내보내기 가이드를 참조하세요.

  • TensorFlow Lite(TFLite)는 온디바이스 추론을 위해 설계된 오픈 소스 딥러닝 프레임워크로, 모바일, 임베디드 및 IoT 디바이스에 YOLO26 모델을 배포하는 데 적합합니다. 주요 이점은 다음과 같습니다.

    • 온디바이스 최적화: 데이터를 로컬에서 처리해 지연 시간을 줄이고 개인정보 보호를 강화합니다.
    • 플랫폼 호환성: Android, iOS, 임베디드 Linux 및 MCU를 지원합니다.
    • 성능: 하드웨어 가속을 사용해 모델의 속도와 효율성을 최적화합니다.

    자세한 내용은 TFLite 가이드를 참조하세요.

  • 예, Raspberry Pi에서 YOLO26 TFLite 모델을 실행해 추론 속도를 높일 수 있습니다. 먼저 위에서 설명한 대로 모델을 LiteRT 형식으로 내보내세요. 그런 다음 Raspberry Pi에서 .tflite 모델을 YOLO("yolo26n.tflite") 또는 LiteRT 인터프리터와 함께 실행하세요.

    Coral Edge TPU를 사용하면 추론을 더욱 가속할 수 있습니다.

  • TFLite는 리소스가 제한된 마이크로컨트롤러에 배포할 수 있습니다. 핵심 런타임은 Arm Cortex M3에서 16 KB의 메모리만 필요하며 여러 기본 모델을 실행할 수 있습니다. 그러나 YOLO26 모델은 일반적으로 일반적인 마이크로컨트롤러 메모리 용량에 비해 너무 크므로, YOLO26에는 단일 보드 컴퓨터, 모바일 디바이스 또는 전용 가속기가 더 적합합니다.

    시작하려면 마이크로컨트롤러용 TFLite Micro 가이드를 참조하세요.

  • TensorFlow Lite는 폭넓은 플랫폼 호환성을 제공하므로 다음을 비롯한 다양한 디바이스에 YOLO26 모델을 배포할 수 있습니다.

    • Android 및 iOS: TFLite Android 및 iOS 라이브러리를 통해 네이티브로 지원됩니다.
    • 임베디드 Linux: Raspberry Pi와 같은 단일 보드 컴퓨터에 적합합니다.
    • 마이크로컨트롤러: 리소스가 제한된 MCU에 적합합니다.

    배포 옵션에 관한 자세한 내용은 배포 가이드를 참조하세요.

  • YOLO26 모델을 LiteRT로 내보내는 동안 오류가 발생하면 다음과 같은 일반적인 해결 방법을 시도해 보세요.

    • 패키지 호환성 확인: Ultralytics, litert-torch 및 ai-edge-litert의 호환되는 버전을 사용하고 있는지 확인하세요. 설치 가이드를 참조하세요.
    • 모델 지원 여부: Ultralytics 내보내기 문서 페이지를 확인하여 해당 YOLO26 모델이 LiteRT 내보내기를 지원하는지 확인하세요.
    • 양자화 문제: INT8 양자화를 사용할 때는 data 매개변수에 데이터셋 경로가 올바르게 지정되어 있는지 확인하세요.

    추가 문제 해결 팁은 일반적인 문제 가이드를 참조하세요.

댓글115

질문을 하거나 결과를 공유하거나 다른 사람을 도와주세요.

  1. saleha-fatimaGitHub(수정됨)

    hi there @glenn-jocher I am trying to convert the the yolov10 model to int8 with tflite format. but same error is occurring again and again. I've tried the solutions described in this thread but nothing seems to work. I am using this for conversion:

    ` from ultralytics import YOLO

    model = YOLO("current_best4.pt")

    model.export(format="tflite", int8=True, data='./datasets/data.yaml', imgsz=640) # creates 'yolov8n_float32.tflite'

    tflite_model = YOLO("yolov8n_int8.tflite")

    results = tflite_model("./frame_365.jpg")

    results.show() `

    and the error i am receiving is this:

    ValueError: Exception encountered when calling layer "tf.math.add_5" (type TFOpLambda).

    Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_5/Add}} = AddV2 [T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,80,160,32], [1,80,160,16].

    Call arguments received by layer "tf.math.add_5" (type TFOpLambda): • x=tf.Tensor(shape=(1, 80, 160, 32), dtype=float32) • y=tf.Tensor(shape=(1, 80, 160, 16), dtype=float32) • name='/model.2/m.0/Add'

    ERROR: input_onnx_file_path: weights/yolov10n.onnx ERROR: onnx_op_name: /model.2/m.0/Add ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again. ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option. ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option. `

    i

    I have approx 5k images for calibration, so thats not the issue, all the packages are upto date as well. can you please suggest what solution should i opt for?

    1
    1. Glenn JocherUltralytics(수정됨)

      @saleha-fatima the error appears to be related to shape mismatches during ONNX to TFLite conversion. Please ensure that your model, dataset, and TensorFlow versions are fully compatible. Refer to the onnx2tf parameter replacement guide for addressing this issue. Additionally, try exporting with int8=False first to confirm the issue is specific to quantization. If the problem persists, please share more details about your environment or test with the latest Ultralytics version.

      Note: Since Ultralytics 8.4.83, format="tflite" is deprecated and exports through LiteRT (litert-torch) instead of onnx2tf and TensorFlow. Run pip install -U ultralytics and export with format="litert" (see the LiteRT guide).

    2. saleha-fatimaGitHub(수정됨)

      I've tried with int8=False, but the problem persists. this is the info related to my model, packages and dataset so i am using yolov10 model fro conversion purposes coco dataset python version=3.9.12 packages:

      torch==2.0.1 torchvision==0.15.2 onnx==1.17.0 onnxruntime==1.15.1 pycocotools==2.0.7 PyYAML==6.0.1 scipy==1.13.0 onnxslim==0.1.31 onnxruntime-gpu==1.18.0 gradio==4.31.5 opencv-python==4.9.0.80 psutil==5.9.8 py-cpuinfo==9.0.0 huggingface-hub==0.23.2 safetensors==0.4.3 ultralytics==8.3.69 onnx2tf==1.20.0 onnxruntime==1.15.1 tensorflow==2.18.0

      I have alse tried the suggestions mentioned in error msg but nothing seems to work!

    3. Glenn JocherUltralytics(수정됨)

      This was a compatibility issue between YOLOv10's architecture and the old ONNX->TFLite (onnx2tf) conversion path. Since Ultralytics 8.4.83, format="tflite" is deprecated and exports through LiteRT (litert-torch) instead of onnx2tf and TensorFlow, so those version pins no longer apply. Run pip install -U ultralytics and export with format="litert" (see the LiteRT guide).

    4. nicola-321GitHub

      hi there @glenn-jocher I am trying to convert the the yolov10 model to int8 with tflite format. but same error is occurring again and again. I've tried the solutions described in this thread but nothing seems to work. I am using this for conversion:

      ` from ultralytics import YOLO

      model = YOLO("current_best4.pt")

      model.export(format="tflite", int8=True, data='./datasets/data.yaml', imgsz=640) # creates 'yolov8n_float32.tflite'

      tflite_model = YOLO("yolov8n_int8.tflite")

      results = tflite_model("./frame_365.jpg")

      results.show() `

      and the error i am receiving is this:

      ERROR: The trace log is below. Traceback (most recent call last): File "/mnt/drive/rev9ai/anaconda3/envs/int8_test/lib/python3.9/site-packages/onnx2tf/utils/common_functions.py", line 310, in print_wrapper_func result = func(*args, **kwargs) File "/mnt/drive/rev9ai/anaconda3/envs/int8_test/lib/python3.9/site-packages/onnx2tf/utils/common_functions.py", line 383, in inverted_operation_enable_disable_wrapper_func result = func(*args, **kwargs) File "/mnt/drive/rev9ai/anaconda3/envs/int8_test/lib/python3.9/site-packages/onnx2tf/utils/common_functions.py", line 53, in get_replacement_parameter_wrapper_func func(*args, **kwargs) File "/mnt/drive/rev9ai/anaconda3/envs/int8_test/lib/python3.9/site-packages/onnx2tf/ops/Add.py", line 248, in make_node merge_two_consecutive_identical_ops_into_one( File "/mnt/drive/rev9ai/anaconda3/envs/int8_test/lib/python3.9/site-packages/onnx2tf/utils/common_functions.py", line 5384, in merge_two_consecutive_identical_ops_into_one tf.math.add( File "/mnt/drive/rev9ai/anaconda3/envs/int8_test/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py", line 142, in wrapper return op(*args, **kwargs) File "/mnt/drive/rev9ai/anaconda3/envs/int8_test/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/mnt/drive/rev9ai/anaconda3/envs/int8_test/lib/python3.9/site-packages/tf_keras/src/layers/core/tf_op_layer.py", line 119, in handle return TFOpLambda(op)(*args, **kwargs) File "/mnt/drive/rev9ai/anaconda3/envs/int8_test/lib/python3.9/site-packages/tf_keras/src/utils/traceback_utils.py", line 70, in error_handler raise e.with_traceback(filtered_tb) from None ValueError: Exception encountered when calling layer "tf.math.add_5" (type TFOpLambda). Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_5/Add}} = AddV2 [T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,80,160,32], [1,80,160,16]. Call arguments received by layer "tf.math.add_5" (type TFOpLambda): • x=tf.Tensor(shape=(1, 80, 160, 32), dtype=float32) • y=tf.Tensor(shape=(1, 80, 160, 16), dtype=float32) • name='/model.2/m.0/Add' ERROR: input_onnx_file_path: weights/yolov10n.onnx ERROR: onnx_op_name: /model.2/m.0/Add ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again. ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option. ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.

      i

      I have approx 5k images for calibration, so thats not the issue, all the packages are upto date as well. can you please suggest what solution should i opt for?

      I've got the same error when covert yolov8 to tflite. Have you fix it?

    5. Glenn JocherUltralytics(수정됨)

      For YOLOv10 TFLite conversion issues in the old onnx2tf path: Since Ultralytics 8.4.83, format="tflite" is deprecated and exports through LiteRT (litert-torch) instead of onnx2tf and TensorFlow, so those version pins no longer apply. Run pip install -U ultralytics and export with format="litert" (see the LiteRT guide). If errors persist, share the full export log.

    6. nicola-321GitHub

      Thanks for your advice. I've tried to Update dependencies and Export with dynamic axes , but unfortunately, a new issue is as below:

      AssertionError: ERROR ❌️ argument 'dynamic' is not supported for format='tflite'

      If I only update dependencies and delete the dynamic axes , the issue is as before:

      ValueError: Exception encountered when calling layer "tf.math.add_5" (type TFOpLambda).

      Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_5/Add}} = AddV2 [T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,80,160,32], [1,80,160,16].

      Call arguments received by layer "tf.math.add_5" (type TFOpLambda): • x=tf.Tensor(shape=(1, 80, 160, 32), dtype=float32) • y=tf.Tensor(shape=(1, 80, 160, 16), dtype=float32) • name='/model.2/m.0/Add'

    7. saleha-fatimaGitHub(수정됨)

      I've got the same error when covert yolov8 to tflite. Have you fix it?

      hi there, yes i fixed this issue. but it then resulted into new error saying "end2end model format conversion is not supported". btw i resolved that issue by using the following versions of dependencies: torch 2.6.0 tensorflow 2.15.1 onnx2tf 1.20.0

      try this but if you are looking for direct export of the model to int8 with tflite. then according to my research export command does not work. atleast it didnt work for yolov10 model. for that i converted the .pt model to IR form (onnx) then further conversion. but if your concern is just to convert model to tflite with fp16/32 quantization. i would suggest to use command line prompt for conversion. for some reasons it works

      !yolo export model=runs/detect/train2/weights/best.pt format=tflite half=False simplify opset=13 workspace=16

    8. Glenn JocherUltralytics(수정됨)

      @saleha-fatima thanks for sharing your solution! Since Ultralytics 8.4.83, format="tflite" is deprecated and exports through LiteRT (litert-torch) instead of onnx2tf and TensorFlow. Run pip install -U ultralytics and export with format="litert" (see the LiteRT guide). The dynamic argument is not supported for LiteRT export.

    9. nicola-321GitHub(수정됨)

      I've got the same error when covert yolov8 to tflite. Have you fix it?

      hi there, yes i fixed this issue. but it then resulted into new error saying "end2end model format conversion is not supported". btw i resolved that issue by using the following versions of dependencies: torch 2.6.0 tensorflow 2.15.1 onnx2tf 1.20.0

      try this but if you are looking for direct export of the model to int8 with tflite. then according to my research export command does not work. atleast it didnt work for yolov10 model. for that i converted the .pt model to IR form (onnx) then further conversion. but if your concern is just to convert model to tflite with fp16/32 quantization. i would suggest to use command line prompt for conversion. for some reasons it works

      !yolo export model=runs/detect/train2/weights/best.pt format=tflite half=False simplify opset=13 workspace=16

      Thank you for your reply,. I've tried to update the dependencies or use the CLI but error persists. so I tried to convert the model step by step . It can be converted to onnx but when converting to tf. An new error occurs as below. It seems like the dependency conflict :

      https://github.com/tensorflow/addons warnings.warn( Traceback (most recent call last): File "/workspace/user_code/model/convertModel/onnx-tensorflow/example/onnx_to_tf.py", line 3, in from onnx_tf.backend import prepare File "/workspace/user_code/model/convertModel/onnx-tensorflow/onnx_tf/init.py", line 1, in from . import backend File "/workspace/user_code/model/convertModel/onnx-tensorflow/onnx_tf/backend.py", line 28, in from onnx_tf.common.handler_helper import get_all_backend_handlers File "/workspace/user_code/model/convertModel/onnx-tensorflow/onnx_tf/common/handler_helper.py", line 3, in from onnx_tf.handlers.backend import * # noqa File "/workspace/user_code/model/convertModel/onnx-tensorflow/onnx_tf/handlers/backend/hardmax.py", line 3, in import tensorflow_addons as tfa File "/workspace/user_code/model/convertModel/myenv3.10/lib/python3.10/site-packages/tensorflow_addons/init.py", line 23, in from tensorflow_addons import activations File "/workspace/user_code/model/convertModel/myenv3.10/lib/python3.10/site-packages/tensorflow_addons/activations/init.py", line 17, in from tensorflow_addons.activations.gelu import gelu File "/workspace/user_code/model/convertModel/myenv3.10/lib/python3.10/site-packages/tensorflow_addons/activations/gelu.py", line 19, in from tensorflow_addons.utils.types import TensorLike File "/workspace/user_code/model/convertModel/myenv3.10/lib/python3.10/site-packages/tensorflow_addons/utils/types.py", line 29, in from keras.src.engine import keras_tensor ModuleNotFoundError: No module named 'keras.src.engine'

      and if I use !pip install 'keras<3.0.0' mediapipe-model-maker to fix the error from https://github.com/google-ai-edge/mediapipe/issues/5229 then it will leading to the issue as below:

      ImportError: This version of TensorFlow Probability requires TensorFlow version >= 2.18; Detected an installation of version 2.15.1. Please upgrade TensorFlow to proceed.

      It seems like that keras<3.0.0 and TensorFlow version >= 2.18 is conflict

    10. saleha-fatimaGitHub

      Hi, so i tested it on my end again and it worked. I would recommend you to create a new env and then just go for installing updated "ultralytics" package using pip install ultralytics now without installing anything else just run this CLI export command:

      yolo export model=runs/detect/train2/weights/best.pt format=tflite half=False simplify opset=13 workspace=16

      It will install all the supported requirements itself. I have tested it, i am sure it will work for u as well. Do update me on this!

    11. Glenn JocherUltralytics(수정됨)

      Thanks for sharing your findings. For TFLite conversion issues, create a fresh environment and let Ultralytics install what it needs when you export. Since Ultralytics 8.4.83, format="tflite" is deprecated and exports through LiteRT (litert-torch) instead of onnx2tf and TensorFlow, so those version pins no longer apply. Run pip install -U ultralytics and export with format="litert" (see the LiteRT guide). If issues persist, please share your environment details from yolo checks.

    12. nicola-321GitHub

      Hi, so i tested it on my end again and it worked. I would recommend you to create a new env and then just go for installing updated "ultralytics" package using pip install ultralytics now without installing anything else just run this CLI export command:

      yolo export model=runs/detect/train2/weights/best.pt format=tflite half=False simplify opset=13 workspace=16

      It will install all the supported requirements itself. I have tested it, i am sure it will work for u as well. Do update me on this!

      Hi, thanks for your sharing!Congratulate that you fix it ! but seems that it does's work in my env. I have create a new env . only "pip install ultralytics" , loading the yolov11. and my python packages is as below: `Package Version


      absl-py 2.1.0 astunparse 1.6.3 cachetools 5.5.2 certifi 2025.1.31 cffi 1.17.1 charset-normalizer 3.4.1 coloredlogs 15.0.1 contourpy 1.1.1 cycler 0.12.1 filelock 3.16.1 flatbuffers 25.2.10 fonttools 4.56.0 fsspec 2025.2.0 gast 0.4.0 google-auth 2.38.0 google-auth-oauthlib 1.0.0 google-pasta 0.2.0 grpcio 1.70.0 h5py 3.11.0 humanfriendly 10.0 idna 3.10 importlib_metadata 8.5.0 importlib_resources 6.4.5 Jinja2 3.1.5 keras 2.13.1 kiwisolver 1.4.7 libclang 18.1.1 Markdown 3.7 MarkupSafe 2.1.5 matplotlib 3.7.5 mpmath 1.3.0 networkx 3.1 numpy 1.24.3 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 9.1.0.70 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu12 2.20.5 nvidia-nvjitlink-cu12 12.8.61 nvidia-nvtx-cu12 12.1.105 oauthlib 3.2.2 onnx 1.17.0 onnx_graphsurgeon 0.5.5 onnx2tf 1.20.0 onnxruntime 1.16.3 onnxslim 0.1.48 opencv-python 4.11.0.86 opt_einsum 3.4.0 packaging 24.2 pandas 2.0.3 pillow 10.4.0 pip 25.0.1 protobuf 5.29.3 psutil 7.0.0 py-cpuinfo 9.0.0 pyasn1 0.6.1 pyasn1_modules 0.4.1 pybind11 2.13.6 pycparser 2.22 pyparsing 3.1.4 python-dateutil 2.9.0.post0 pytz 2025.1 PyYAML 6.0.2 requests 2.32.3 requests-oauthlib 2.0.0 rsa 4.9 scipy 1.10.1 seaborn 0.13.2 setuptools 49.2.1 six 1.17.0 sng4onnx 1.0.4 sounddevice 0.5.1 sympy 1.13.3 tensorboard 2.13.0 tensorboard-data-server 0.7.2 tensorflow-cpu 2.13.1 tensorflow-estimator 2.13.0 tensorflow-io-gcs-filesystem 0.34.0 termcolor 2.4.0 tf-keras 2.15.0 tflite-support 0.4.0 torch 2.4.1 torchvision 0.19.1 tqdm 4.67.1 triton 3.0.0 typing_extensions 4.5.0 tzdata 2025.1 ultralytics 8.3.78 ultralytics-thop 2.0.14 urllib3 2.2.3 Werkzeug 3.0.6 wheel 0.45.1 wrapt 1.17.2 zipp 3.20.2`

    13. nicola-321GitHub(수정됨)

      The problem is still `ValueError: Exception encountered when calling layer "tf.math.add_5" (type TFOpLambda).

      Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_5/Add}} = AddV2 [T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,16,160,32], [1,16,160,16].

      Call arguments received by layer "tf.math.add_5" (type TFOpLambda): • x=tf.Tensor(shape=(1, 16, 160, 32), dtype=float32) • y=tf.Tensor(shape=(1, 16, 160, 16), dtype=float32) • name='/model.2/m.0/Add'

      ERROR: input_onnx_file_path: yolo11n.onnx ERROR: onnx_op_name: /model.2/m.0/Add ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again. ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option. ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.`

    14. Glenn JocherUltralytics(수정됨)

      Thanks for sharing your environment details. Since Ultralytics 8.4.83, format="tflite" is deprecated and exports through LiteRT (litert-torch) instead of onnx2tf and TensorFlow, so those version pins no longer apply. Run pip install -U ultralytics and export with format="litert" (see the LiteRT guide). Then export via CLI: yolo export model=yolo11n.pt format=litert imgsz=640.

  2. Z-Mike0033GitHub

    您好,我现在正在使用基于官网的yolo11m.pt预训练模型,在自己数据集上训练后的human_detector_model_float32.tflite,并在支持 TFLite GPU 代理的设备上进行推理。 1、从.pt到tflite的转换代码是官网提供的版本如下: model.export( format="tflite", imgsz=640, half=True, simplify=True ) 2、使用tflite在支持 TFLite GPU 代理的设备上进行推理,日志如下: I human_detector_mode_float32.tflite I Created GPU engine. INFO: Created TensorFlow Lite delegate for GPU. ERROR: Failed to build program executable - Build program failure error: Total size of kernel arguments exceeds limit! Total arguments size: 4120, limit: 2048 in kernel: 'main_function' error: backend compiler failed build.

    error: Total size of kernel arguments exceeds limit! Total arguments size: 4120, limit: 2048 in kernel: 'main_function' error: backend compiler failed build.

    ERROR: Falling back to OpenGL ERROR: TfLiteGpuDelegate Init: OpenGL-based API disabled ERROR: TfLiteGpuDelegate Prepare: delegate is not initialized ERROR: Node number 255 (TfLiteGpuDelegateV2) failed to prepare. ERROR: Restored original execution plan after delegate application failure. E tflite gpu not support!../models/extra/human_detector/human_detector_model_float32.tflite INFO: Created TensorFlow Lite XNNPACK delegate for CPU. I Load model finished I Image reader initialized. Total images: 15 I Starting image processing loop.. 3、由日志可知,模型成功加载,并创建了 GPU 代理;但在准备阶段,GPU 代理失败了,并回退到 CPU(XNNPACK)推理。主要原因是“Total size of kernel arguments exceeds limit!:**内核参数总大小超过了硬件限制。**日志显示参数大小为 4120,而限制为 2048”。 4、想问下这个是为什么?需要怎么解决这个GPU跑不了回退xnnpack的问题?

  3. UltralyticsAssistantGitHubUltralytics

    👋 Hello, thank you for your interest in Ultralytics 🚀! For users exploring model conversion and deployment, we recommend checking out the Docs where you’ll find a variety of helpful Python and CLI usage examples, as well as detailed integration guides.

    If this is a 🐛 Bug Report regarding TFLite conversion, please provide a minimum reproducible example to help us investigate.

    If you have a custom training ❓ Question or experience any issues during conversion, please include as much information as possible, such as code snippets, error messages, and details about your environment. Also, check our Tips for Best Training Results to ensure optimal performance.

    Connect with the Ultralytics community in the way that suits you best: for real-time chat, join us on Discord 🎧, dive deeper on Discourse, or join discussions on our Subreddit.

    Upgrade

    Please upgrade to the latest ultralytics package and all requirements in a Python>=3.8 environment with PyTorch>=1.8 to ensure your issue is not already resolved:

    pip install -U ultralytics

    Environments

    YOLO can be run in any of these up-to-date verified environments (with all dependencies such as CUDA/CUDNN, Python and PyTorch preinstalled):

    Status

    Ultralytics CI

    If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLO Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

    🔔 This is an automated response. An Ultralytics engineer will also assist you here soon!

    1. Alexts166GitHub

      I'm working through the IDE, and I've updated the dependencies. I'm launching it from ultralytics import YOLO

      Load the YOLO11 model

      model = YOLO("yolo11m-obb.pt")

      Export the model to TFLite format

      model.export(format="tflite")

      ERROR TensorFlow SavedModel: export failure 34.3s: No module named 'ai_edge_litert' Traceback (most recent call last): File "D:\nEWpapka\convert.py", line 7, in model.export(format="tflite") # creates 'yolo11n_float32.tflite' File "D:\nEWpapka.venv\lib\site-packages\ultralytics\engine\model.py", line 734, in export return Exporter(overrides=args, callbacks=self.callbacks)(model=self.model) File "D:\nEWpapka.venv\lib\site-packages\ultralytics\engine\exporter.py", line 494, in call f[5], keras_model = self.export_saved_model() File "D:\nEWpapka.venv\lib\site-packages\ultralytics\engine\exporter.py", line 198, in outer_func raise e File "D:\nEWpapka.venv\lib\site-packages\ultralytics\engine\exporter.py", line 193, in outer_func f, model = inner_func(*args, **kwargs) File "D:\nEWpapka.venv\lib\site-packages\ultralytics\engine\exporter.py", line 1003, in export_saved_model import onnx2tf # scoped for after ONNX export for reduced conflict during import File "D:\nEWpapka.venv\lib\site-packages\onnx2tf_init.py", line 1, in from onnx2tf.onnx2tf import convert, main File "D:\nEWpapka.venv\lib\site-packages\onnx2tf\onnx2tf.py", line 44, in from onnx2tf.utils.common_functions import ( File "D:\nEWpapka.venv\lib\site-packages\onnx2tf\utils\common_functions.py", line 20, in from ai_edge_litert.interpreter import Interpreter ModuleNotFoundError: No module named 'ai_edge_litert'

      in bash pip install ai_edge_litert ERROR: Could not find a version that satisfies the requirement ai_edge_litert (from versions: none) ERROR: No matching distribution found for ai_edge_litert

    2. Glenn JocherUltralytics

      You're correct that ai_edge_litert has limited platform support and isn't available for Windows. For TFLite export on Windows, I recommend using one of these alternatives:

      1. Google Colab (recommended): Export your model in a Colab notebook which runs on Linux
      2. Docker: Use the Ultralytics Docker image on Windows to export in a Linux container
      3. WSL2: Export using Windows Subsystem for Linux

      The simplest approach is Colab - just upload your .pt file and run the export code there, then download the resulting .tflite file back to your Windows machine.

  4. Alexts166GitHub

    Hi! Please give me a hint. Exporting the model .pt in the model . tflite according to the code in the documentation. It remains to cope with the ai_edge_litert package. Do I understand correctly that there is no such package for WINDOWS? based on the information on https://pypi.org/project/ai-edge-litert/#files Or is there a way out?

  5. longzaixiang222GitHub

    您好老师,同一个模型,转换成tflite,同一张图片,每2秒识别一次,和单次识别为什么结果会不样?

    1. Glenn JocherUltralytics

      @longzaixiang222 这种差异通常由以下原因造成:1) 录屏图像可能存在压缩损失或格式转换;2) 录屏帧和保存的图片可能在色彩空间、分辨率或编码上有细微差别;3) 图像预处理路径不同。建议检查两种图像的像素值、尺寸和格式是否完全一致,确保使用相同的预处理步骤。您可以尝试保存录屏帧并与本地图片进行像素级对比来定位差异源。

  6. diamondeyessGitHub

    我的导出设置是:ormat = "tflite", device = "0", int8 = True, data =.....data.yaml, 为什么输出:fully_quantize: 0, inference_type: 6, input_inference_type: FLOAT32, output_inference_type: FLOAT32 ,没有完全量化吗?

    1. Glenn JocherUltralytics

      @diamondeyess 这是正常的TFLite INT8量化行为!输入输出保持FLOAT32是为了保持精度和兼容性,而模型内部计算已经量化为INT8以提升性能和减小模型大小。这种"混合量化"可以在保持推理精度的同时获得INT8的性能优势。如需完全INT8量化(包括输入输出),需要特定的TFLite配置,但通常不推荐,因为可能影响精度。

    2. diamondeyessGitHub

      好的,谢谢!后面输出还有一段:fully_quantize: 0, inference_type: 6, input_inference_type: INT8, output_inference_type: INT8 就是您说的完全量化吗?

    3. Glenn JocherUltralytics

      @diamondeyess 是的,第二个输出(input_inference_type: INT8, output_inference_type: INT8)就是完全量化!这意味着模型的输入和输出也被量化为INT8。完全量化可以进一步减小模型大小和提升推理速度,但需要注意输入数据的预处理要匹配INT8格式。在使用时,您可能需要将输入数据从FLOAT32转换为INT8范围,并在推理后将输出从INT8转换回FLOAT32以便后处理。

  7. SKUNAandGOJOGitHub

    I'm trying to convert "tflite" to Google Colab, but it's showing like this

    Ultralytics 8.3.133 🚀 Python-3.11.12 torch-2.6.0+cu124 CPU (Intel Xeon 2.00GHz) YOLO11n summary (fused): 100 layers, 2,584,687 parameters, 0 gradients, 6.3 GFLOPs

    PyTorch: starting from 'fruit.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 17, 8400) (5.2 MB) requirements: Ultralytics requirements ['sng4onnx>=1.0.1', 'onnx_graphsurgeon>=0.3.26', 'ai-edge-litert>=1.2.0', 'onnx2tf>=1.26.3', 'protobuf>=5'] not found, attempting AutoUpdate... Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Collecting sng4onnx>=1.0.1 Downloading sng4onnx-1.0.4-py3-none-any.whl.metadata (4.6 kB) Collecting onnx_graphsurgeon>=0.3.26 Downloading onnx_graphsurgeon-0.5.8-py2.py3-none-any.whl.metadata (8.2 kB) Collecting ai-edge-litert>=1.2.0 Downloading ai_edge_litert-1.2.0-cp311-cp311-manylinux_2_17_x86_64.whl.metadata (1.6 kB) Collecting onnx2tf>=1.26.3 Downloading onnx2tf-1.27.4-py3-none-any.whl.metadata (147 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 147.7/147.7 kB 8.2 MB/s eta 0:00:00 Collecting protobuf>=5 Downloading protobuf-6.30.2-cp39-abi3-manylinux2014_x86_64.whl.metadata (593 bytes) Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (from onnx_graphsurgeon>=0.3.26) (2.0.2) Requirement already satisfied: onnx>=1.14.0 in /usr/local/lib/python3.11/dist-packages (from onnx_graphsurgeon>=0.3.26) (1.17.0) Requirement already satisfied: flatbuffers in /usr/local/lib/python3.11/dist-packages (from ai-edge-litert>=1.2.0) (25.2.10) Downloading sng4onnx-1.0.4-py3-none-any.whl (5.9 kB) Downloading onnx_graphsurgeon-0.5.8-py2.py3-none-any.whl (57 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.9/57.9 kB 280.1 MB/s eta 0:00:00 Downloading ai_edge_litert-1.2.0-cp311-cp311-manylinux_2_17_x86_64.whl (3.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.5/3.5 MB 103.4 MB/s eta 0:00:00 Downloading onnx2tf-1.27.4-py3-none-any.whl (446 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 446.1/446.1 kB 379.6 MB/s eta 0:00:00 Downloading protobuf-6.30.2-cp39-abi3-manylinux2014_x86_64.whl (316 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 316.2/316.2 kB 356.0 MB/s eta 0:00:00 Installing collected packages: sng4onnx, protobuf, onnx2tf, ai-edge-litert, onnx_graphsurgeon Attempting uninstall: protobuf Found existing installation: protobuf 3.20.3 Uninstalling protobuf-3.20.3: Successfully uninstalled protobuf-3.20.3 Successfully installed ai-edge-litert-1.2.0 onnx2tf-1.27.4 onnx_graphsurgeon-0.5.8 protobuf-6.30.2 sng4onnx-1.0.4

    requirements: AutoUpdate success ✅ 13.1s, installed 5 packages: ['sng4onnx>=1.0.1', 'onnx_graphsurgeon>=0.3.26', 'ai-edge-litert>=1.2.0', 'onnx2tf>=1.26.3', 'protobuf>=5'] WARNING ⚠️ requirements: Restart runtime or rerun command for updates to take effect

    TensorFlow SavedModel: starting export with tensorflow 2.18.0... Downloading https://github.com/ultralytics/assets/releases/download/v8.3.0/calibration_image_sample_data_20x128x128x3_float32.npy.zip to 'calibration_image_sample_data_20x128x128x3_float32.npy.zip'... 100%|██████████| 1.11M/1.11M [00:00<00:00, 25.9MB/s] Unzipping calibration_image_sample_data_20x128x128x3_float32.npy.zip to /content/calibration_image_sample_data_20x128x128x3_float32.npy...: 100%|██████████| 1/1 [00:00<00:00, 32.58file/s] ONNX: starting export with onnx 1.18.0 opset 19...

    ONNX: slimming with onnxslim 0.1.52... ONNX: export success ✅ 1.7s, saved as 'fruit.onnx' (10.2 MB) TensorFlow SavedModel: starting TFLite export with onnx2tf 1.27.4... ERROR:root:Internal Python error in the inspect module. Below is the traceback from this internal error.

    ERROR: The trace log is below. Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 313, in print_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 386, in inverted_operation_enable_disable_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 56, in get_replacement_parameter_wrapper_func func(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Add.py", line 281, in make_node merge_two_consecutive_identical_ops_into_one( File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 5475, in merge_two_consecutive_identical_ops_into_one tf.math.add( File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/ops/weak_tensor_ops.py", line 142, in wrapper return op(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/layers/core/tf_op_layer.py", line 119, in handle return TFOpLambda(op)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/utils/traceback_utils.py", line 72, in error_handler del filtered_tb ValueError: Exception encountered when calling layer "tf.math.add_17" (type TFOpLambda).

    Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_17/Add}} = AddV2 [T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,16,160,32], [1,16,160,16].

    Call arguments received by layer "tf.math.add_17" (type TFOpLambda): • x=tf.Tensor(shape=(1, 16, 160, 32), dtype=float32) • y=tf.Tensor(shape=(1, 16, 160, 16), dtype=float32) • name='wa/model.2/m.0/Add'

    ERROR: input_onnx_file_path: fruit.onnx ERROR: onnx_op_name: wa/model.2/m.0/Add ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again. ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option. ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option. Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 313, in print_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 386, in inverted_operation_enable_disable_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 56, in get_replacement_parameter_wrapper_func func(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Add.py", line 281, in make_node merge_two_consecutive_identical_ops_into_one( File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 5475, in merge_two_consecutive_identical_ops_into_one tf.math.add( File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/ops/weak_tensor_ops.py", line 142, in wrapper return op(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/layers/core/tf_op_layer.py", line 119, in handle return TFOpLambda(op)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/utils/traceback_utils.py", line 72, in error_handler del filtered_tb ValueError: Exception encountered when calling layer "tf.math.add_17" (type TFOpLambda).

    Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_17/Add}} = AddV2 [T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,16,160,32], [1,16,160,16].

    Call arguments received by layer "tf.math.add_17" (type TFOpLambda): • x=tf.Tensor(shape=(1, 16, 160, 32), dtype=float32) • y=tf.Tensor(shape=(1, 16, 160, 16), dtype=float32) • name='wa/model.2/m.0/Add'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/IPython/core/interactiveshell.py", line 3553, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 7, in <cell line: 0> model.export(format="tflite") File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/model.py", line 730, in export return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/exporter.py", line 468, in call f[5], keras_model = self.export_saved_model() ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/exporter.py", line 192, in outer_func f, model = inner_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/exporter.py", line 975, in export_saved_model keras_model = onnx2tf.convert( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/onnx2tf.py", line 1149, in convert op.make_node( File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 379, in print_wrapper_func sys.exit(1) SystemExit: 1

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py", line 1101, in get_records return _fixed_getinnerframes(etb, number_of_lines_of_context, tb_offset) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py", line 248, in wrapped return f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py", line 281, in _fixed_getinnerframes records = fix_frame_records_filenames(inspect.getinnerframes(etb, context)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/inspect.py", line 1739, in getinnerframes traceback_info = getframeinfo(tb, context) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/inspect.py", line 1671, in getframeinfo lineno = frame.f_lineno ^^^^^^^^^^^^^^ AttributeError: 'tuple' object has no attribute 'f_lineno'

    ValueError Traceback (most recent call last) /usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py in print_wrapper_func(*args, **kwargs) 312 try: --> 313 result = func(*args, **kwargs) 314

    22 frames ValueError: Exception encountered when calling layer "tf.math.add_17" (type TFOpLambda).

    Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_17/Add}} = AddV2 [T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,16,160,32], [1,16,160,16].

    Call arguments received by layer "tf.math.add_17" (type TFOpLambda): • x=tf.Tensor(shape=(1, 16, 160, 32), dtype=float32) • y=tf.Tensor(shape=(1, 16, 160, 16), dtype=float32) • name='wa/model.2/m.0/Add'

    During handling of the above exception, another exception occurred:

    SystemExit Traceback (most recent call last) [... skipping hidden 1 frame]

    SystemExit: 1

    During handling of the above exception, another exception occurred:

    TypeError Traceback (most recent call last) [... skipping hidden 1 frame]

    /usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py in find_recursion(etype, value, records) 380 # first frame (from in to out) that looks different. 381 if not is_recursion_error(etype, value, records): --> 382 return len(records), 0 383 384 # Select filename, lineno, func_name to track frames with

    TypeError: object of type 'NoneType' has no len()

  8. SidyeiGitHub

    I directly exported my yolov8s trained model (best.pt) to tensorflow lite using this block of code:

    from ultralytics import YOLO

    model = YOLO("/content/runs/detect/yolov8s_sign_language/weights/best.pt") model.export( format="tflite", # Export to TFLite imgsz=[320, 320], # Match training input size dynamic=False, # Fixed batch size (required for TFLite) int8=False, # Disable quantization (preserves FP32 precision) simplify=True, # Optimize model (removes redundant ops) opset=12 # ONNX opset (if intermediate conversion is needed) )

    The problem lies after conversion where there was a significant decrease of precision and it detects randomly compared to my best.pt file that performs well. Is this normal? how can I ensure that there are no lose ends during conversion?

  9. McGillllGitHub

    PyTorch: starting from 'my_model.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 6, 8400) (17.6 MB) TensorFlow SavedModel: export failure 0.0s: Unable to convert function return value to a Python type! The signature was () -> handle

    TypeError Traceback (most recent call last) Cell In[11], line 2 1 model = YOLO("my_model.pt")
    ----> 2 model.export(format="tflite")

    1. Glenn JocherUltralytics(수정됨)

      Hi @McGillll,

      The error you're encountering during TFLite export is happening at the TensorFlow SavedModel stage, which is a prerequisite step for TFLite conversion. This function return value conversion error typically stems from version incompatibilities between PyTorch, TensorFlow, and the conversion tools.

      Since Ultralytics 8.4.83, format="tflite" is deprecated and exports through LiteRT (litert-torch) instead of onnx2tf and TensorFlow, so those version pins no longer apply. Run pip install -U ultralytics and export with format="litert" (see the LiteRT guide). Then try the CLI instead of the Python API:

      yolo export model=my_model.pt format=litert imgsz=640

      Let me know if these suggestions help or if you need further assistance!

  10. longzaixiang222GitHub

    from ultralytics import YOLO model = YOLO("best5.pt") model.export(format="tflite") 模型导出后识别效果不好 使用YOLO("best5_float32.tflite") 这种加载方式,效果就很好

    但我使用: interpreter = tf.lite.Interpreter(model_path="best5_float32.tflite") interpreter.allocate_tensors() input_details = interpreter.get_input_details() output_details = interpreter.get_output_details() 识别效果不佳,部分类型识别不出来,原图尺寸640*640

    请问下是您们内部对图片做了什么处理吗?

    1. Glenn JocherUltralytics(수정됨)

      @longzaixiang222 thank you for your question! The difference occurs because when using YOLO("best5_float32.tflite"), we automatically handle preprocessing including image scaling, normalization (pixel values /255), and channel reordering (BGR→RGB). When using the TFLite Interpreter directly, you'll need to manually apply these preprocessing steps.

      For implementation details, please refer to our TFLite export documentation which explains the required input normalization and channel order. Ensure your input tensor matches the model's input shape (for current LiteRT exports (1,3,640,640) NCHW, for older onnx2tf exports (1,640,640,3) NHWC) with float32 dtype and normalized values; the YOLO class also letterboxes the image rather than stretching it.

    2. Glenn JocherUltralytics(수정됨)

      Great observation! When using the TFLite Interpreter directly, you'll need to manually replicate the preprocessing steps our YOLO class handles automatically. Ensure you're resizing images to the export size, normalizing pixel values to 0-1 (divide by 255), and converting BGR→RGB before inference. The input tensor should match the model's input shape (for current LiteRT exports (1,3,640,640) NCHW, for older onnx2tf exports (1,640,640,3) NHWC) with float32 dtype. Resize by letterboxing (padding to keep aspect ratio) as YOLO does. For implementation details, refer to our TFLite preprocessing requirements.