YOLO26 모델 형식에서 TF.js 모델 형식으로 내보내기(지원 중단됨)

지원 중단됨 — LiteRT로 대체됨

Ultralytics 8.4.83부터 tfjs (TensorFlow.js) 내보내기 형식은 더 이상 사용되지 않으며 통합 Google LiteRT 형식으로 대체되었습니다. LiteRT는 LiteRT.js를 통해 브라우저에서 실행되며(WebGPU 가속 지원), TF.js가 지원했던 브라우저 내 및 Node.js 사용 사례뿐 아니라 모바일, 임베디드, 엣지 환경까지 단일 .tflite 모델로 지원합니다.

format="tfjs"은 계속 작동하지만 사용 중단 경고를 표시하고 대신 LiteRT 모델을 내보냅니다. 앞으로는 format="litert"을 사용하세요. 최신 내보내기 지침과 브라우저 배포 경로는 LiteRT 내보내기 가이드를 참조하세요.

머신러닝 모델을 브라우저나 Node.js에서 직접 배포하는 일은 까다로울 수 있습니다. 사용자의 장치에서 대화형 애플리케이션을 로컬로 실행할 수 있도록 모델 형식이 더 빠른 성능에 최적화되어 있는지 확인해야 합니다. TensorFlow.js 또는 TF.js 모델 형식은 전력 소비를 최소화하면서 빠른 성능을 제공하도록 설계되었습니다.

TF.js로 내보내야 하는 이유는 무엇인가요?#

TensorFlow 생태계의 일부로 TensorFlow 팀이 개발한 TensorFlow.js로 머신러닝 모델을 내보내면 머신러닝 애플리케이션 배포에 여러 이점이 있습니다. 민감한 데이터를 장치에 보관하여 사용자의 개인정보 보호와 보안을 강화할 수 있습니다. 아래 이미지는 TensorFlow.js 아키텍처와 머신러닝 모델이 웹 브라우저 및 Node.js에서 변환되고 배포되는 방식을 보여줍니다.

TensorFlow.js browser ML inference architecture

모델을 로컬에서 실행하면 지연 시간이 줄어들어 사용자 경험의 응답성이 향상됩니다. TensorFlow.js는 오프라인 기능도 제공하므로 인터넷에 연결되어 있지 않아도 애플리케이션을 사용할 수 있습니다. TF.js는 확장성을 고려해 설계되었으며 GPU 가속을 지원하므로 리소스가 제한된 기기에서도 복잡한 모델을 효율적으로 실행할 수 있습니다.

TF.js의 주요 기능#

TF.js가 개발자에게 유용한 주요 기능은 다음과 같습니다.

  • 크로스 플랫폼 지원: TensorFlow.js는 브라우저 및 Node.js 환경 모두에서 사용할 수 있어 다양한 플랫폼에 유연하게 배포할 수 있습니다. 이를 통해 개발자는 애플리케이션을 더 쉽게 개발하고 배포할 수 있습니다.

  • 다중 백엔드 지원: TensorFlow.js는 CPU, GPU 가속을 위한 WebGL, 네이티브에 가까운 실행 속도를 제공하는 WebAssembly(WASM), 고급 브라우저 기반 머신러닝 기능을 제공하는 WebGPU 등 다양한 계산 백엔드를 지원합니다.

  • 오프라인 기능: TensorFlow.js를 사용하면 인터넷 연결 없이 브라우저에서 모델을 실행할 수 있으므로 오프라인에서도 작동하는 애플리케이션을 개발할 수 있습니다.

TensorFlow.js를 사용한 배포 옵션#

TF.js는 머신러닝 모델 배포를 위한 다양한 옵션을 제공합니다:

  • 브라우저 내 ML 애플리케이션: 브라우저에서 머신러닝 모델을 직접 실행하는 웹 애플리케이션을 구축할 수 있습니다. 서버 측 계산이 필요하지 않으므로 서버 부하가 줄어듭니다.

  • Node.js 애플리케이션: TensorFlow.js는 Node.js 환경에 배포하는 것도 지원하므로 서버 측 머신러닝 애플리케이션을 개발할 수 있습니다. 서버의 처리 성능이나 서버 측 데이터에 대한 액세스가 필요한 애플리케이션에 특히 유용합니다.

  • Chrome 확장 프로그램: TensorFlow.js를 사용해 Chrome 확장 프로그램을 만드는 것도 흥미로운 배포 시나리오입니다. 예를 들어, 사용자가 웹페이지의 이미지에서 마우스 오른쪽 버튼을 클릭하면 사전 학습된 ML 모델로 이미지를 분류하는 확장 프로그램을 개발할 수 있습니다. TensorFlow.js를 일상적인 웹 브라우징 환경에 통합하면 머신러닝을 바탕으로 즉각적인 인사이트나 기능 향상을 제공할 수 있습니다.

TF.js 내보내기를 LiteRT로 대체#

새로 내보내는 경우 모델을 LiteRT로 변환합니다.

설치#

필수 패키지를 설치하려면 다음을 실행하세요:

설치
# Install the required package for YOLO26
pip install ultralytics

설치 절차에 대한 자세한 지침과 모범 사례는 Ultralytics 설치 가이드를 참조하세요. YOLO26에 필요한 패키지를 설치하는 중 문제가 발생하면 해결 방법과 팁을 확인할 수 있는 일반적인 문제 가이드를 참조하세요.

사용법#

모든 Ultralytics YOLO26 모델은 별도의 설정 없이 내보내기를 지원하도록 설계되어 있어 원하는 배포 워크플로에 쉽게 통합할 수 있습니다. 애플리케이션에 가장 적합한 설정을 선택하려면 지원되는 내보내기 형식 및 구성 옵션 전체 목록을 확인하세요.

tfjs 인수는 이제 LiteRT .tflite 모델을 내보냅니다. 이 모델은 로컬에서 내보내기, 예측, 검증 모드를 지원하며 LiteRT.js를 통해 브라우저에서 실행됩니다.

내보내기
from ultralytics import YOLO

# YOLO26 모델 로드
model = YOLO("yolo26n.pt")

# TF.js를 대체하는 LiteRT 형식으로 모델 내보내기
model.export(format="litert", imgsz=640)  # 분류에는 imgsz=224 사용
예측 및 검증

내보낸 LiteRT .tflite 모델은 yolo predict 및 yolo val로 직접 로드되며, LiteRT.js를 통해 브라우저에서 실행됩니다.

내보내기 인수#

LiteRT 내보내기 인수를 참조하세요. 내보내기 프로세스에 관한 자세한 내용은 내보내기에 관한 Ultralytics 문서 페이지를 참조하세요.

내보낸 YOLO26 TensorFlow.js 모델 배포#

YOLO26 모델을 내보냈다면 다음 단계는 배포입니다. 내보내기를 통해 온디바이스에서 실행되고 LiteRT.js를 사용해 브라우저에서도 실행되는 LiteRT .tflite 모델을 생성합니다.

참고로 기존 TensorFlow.js 런타임 리소스는 다음과 같습니다:

요약#

이 가이드는 기존 TF.js 배포 맥락을 다룹니다. 새로 내보낼 때는 LiteRT를 사용하세요.

사용 방법에 관한 자세한 내용은 TensorFlow.js 공식 문서를 참조하세요.

자주 묻는 질문#

  • 대신 LiteRT로 내보냅니다. format="tfjs"도 계속 작동하지만 지원 중단 경고를 표시하고 동일한 LiteRT 모델을 내보냅니다:

    사용법
    from ultralytics import YOLO
    
    # YOLO26 모델 로드
    model = YOLO("yolo26n.pt")
    
    # TF.js를 대체하는 LiteRT 형식으로 모델 내보내기
    model.export(format="litert", imgsz=640)  # 분류에는 imgsz=224 사용
    
    # 내보낸 '.tflite' 모델은 LiteRT.js를 통해 브라우저에서 실행하거나 yolo predict/val을 사용해 로컬에서 실행할 수 있습니다.

    지원되는 내보내기 옵션에 관한 자세한 내용은 배포 옵션에 관한 Ultralytics 문서 페이지를 참조하세요.

  • TensorFlow.js는 브라우저와 Node.js 환경에서 ML 모델을 효율적으로 실행하도록 특별히 설계되었습니다. 브라우저 기반 애플리케이션에 제공하는 이점은 다음과 같습니다:

    • 지연 시간 감소: 머신러닝 모델을 로컬에서 실행하므로 서버 측 계산에 의존하지 않고 즉각적인 결과를 제공합니다.
    • 개인정보 보호 강화: 민감한 데이터를 사용자의 장치에 보관하여 보안 위험을 최소화합니다.
    • 오프라인 사용 지원: 인터넷 연결 없이도 모델을 실행할 수 있어 일관된 기능을 제공합니다.
    • 다중 백엔드 지원: CPU, WebGL, WebAssembly(WASM), WebGPU 등의 백엔드를 제공하여 다양한 계산 요구 사항에 유연하게 대응합니다.

    TF.js에 대해 더 알고 싶으신가요? TensorFlow.js 공식 가이드를 확인하세요.

댓글45

질문을 하거나 결과를 공유하거나 다른 사람을 도와주세요.

  1. jupiter-mzaGitHub

    Hi, iam trying use yolov8n_web_model in my REACT project, all works fine, the model was loaded, predictions works, but the result, even making conversions stay wrong, its returned from my predictions: { "kept": false, "isDisposedInternal": false, "shape": [ 1, 84, 8400 ], "dtype": "float32", "size": 705600, "strides": [ 705600, 8400 ], "dataId": { "id": 1030 }, "id": 719, "rankType": "3", "scopeId": 575 } when i try to catch the results using this: const handleIdentify = async () => { if (!model) { console.warn('Modelo não carregado.'); return; } const canvas = canvasRef.current; const imageElement = canvas; const tensor = tf.browser.fromPixels(imageElement).expandDims(0).toFloat().div(tf.scalar(255)); const predictions = await model.executeAsync(tensor); // console.log("Predictions:", predictions); const result = await predictions.array(); console.log("Result:", result); // // Parâmetros da sua imagem const imgWidth = 640;
    const imgHeight = 480;

    // Array para armazenar deteções filtradas
    const detections = [];
    
    for (let i = 0; i < result[0].length; i++) {
      const prediction = result[0][i]; // vetor de 84 elementos
    
      const x = prediction[0];
      const y = prediction[1];
      const w = prediction[2];
      const h = prediction[3];
      const objConfidence = prediction[4];
      const classProbs = prediction.slice(5); // de 5 a 83
    
      // Encontra a classe com maior probabilidade
      const maxClassProb = Math.max(...classProbs);
      const classIndex = classProbs.indexOf(maxClassProb);
    
      // Calcula a confiança do objeto
      const confidence = objConfidence * maxClassProb;
    
      // Define um limiar de confiança
      const confidenceThreshold = 0.25;
      if (confidence > confidenceThreshold) {
        // Escala as coordenadas para o tamanho da imagem
        const bx = x * imgWidth;
        const by = y * imgHeight;
        const bw = w * imgWidth;
        const bh = h * imgHeight;
    
        detections.push({
          bbox: [bx - bw / 2, by - bh / 2, bw, bh],
          confidence: confidence,
          class: classIndex,
        });
      }
      console.log("Detections:", detections)
    
    };

    } its return a resultset completely wrong like this: [ { "bbox": [ -2065.0027465820312, -2358.991241455078, 12443.428955078125, 14857.036743164062 ], "confidence": 23974.878087386023, "class": 2154 }, { "bbox": [ 2059.2428588867188, 1746.5789794921875, 2604.3841552734375, 2417.2320556640625 ], "confidence": 2756.7185883242637, "class": 6330 }, { "bbox": [ 3684.2034912109375, 1313.7918090820312, 8792.0166015625, 16787.200927734375 ], "confidence": 18003.25029833475, "class": 8224 }, { "bbox": [ 4112.9534912109375, 3463.8272094726562, 5174.500732421875, 4886.737060546875 ], "confidence": 5542.858029322, "class": 8182 } ] why ?

  2. SarvagyaVaishGitHub

    What version of ultralytics was the tfjs export tested with?

    I am running this minimal example to convert a model to tfjs.

    from ultralytics import YOLO
    
    model = YOLO("yolo11l.pt")
    model.export(format='tfjs')

    Error i am seeing the following error:

    Ultralytics 8.3.133 🚀 Python-3.11.12 torch-2.6.0+cu124 CPU (Intel Xeon 2.20GHz)
    YOLO11l summary (fused): 190 layers, 25,340,992 parameters, 0 gradients, 86.9 GFLOPs
    
    PyTorch: starting from 'yolo11l.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (49.0 MB)
    
    TensorFlow SavedModel: starting export with tensorflow 2.18.0...
    
    ONNX: starting export with onnx 1.18.0 opset 19...
    ONNX: slimming with onnxslim 0.1.52...
    ONNX: export success ✅ 4.4s, saved as 'yolo11l.onnx' (97.0 MB)
    TensorFlow SavedModel: starting TFLite export with onnx2tf 1.27.4...
    ERROR:root:Internal Python error in the inspect module.
    Below is the traceback from this internal error.
    
    ERROR: The trace log is below.
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 313, in print_wrapper_func
        result = func(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 386, in inverted_operation_enable_disable_wrapper_func
        result = func(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 56, in get_replacement_parameter_wrapper_func
        func(*args, **kwargs)
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Conv.py", line 459, in make_node
        conv_bias(
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Conv.py", line 308, in conv_bias
        tf.nn.convolution(
      File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
        raise e.with_traceback(filtered_tb) from None
      File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/layers/core/tf_op_layer.py", line 119, in handle
        return TFOpLambda(op)(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/utils/traceback_utils.py", line 72, in error_handler
        del filtered_tb
    ValueError: Exception encountered when calling layer "tf.nn.convolution_35" (type TFOpLambda).
    
    Depth of input (384) is not a multiple of input depth of filter (256) for '{{node tf.nn.convolution_35/convolution}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](Placeholder, tf.nn.convolution_35/convolution/filter)' with input shapes: [1,64,160,384], [1,1,256,256].
    
    Call arguments received by layer "tf.nn.convolution_35" (type TFOpLambda):
      • input=tf.Tensor(shape=(1, 64, 160, 384), dtype=float32)
      • filters=tf.Tensor(shape=(1, 1, 256, 256), dtype=float32)
      • strides=['1', '1']
      • padding='SAME'
      • data_format=None
      • dilations=['1', '1']
      • name=None
    
    ERROR: input_onnx_file_path: yolo11l.onnx
    ERROR: onnx_op_name: wa/model.2/cv2/conv/Conv
    ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement
    ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again.
    ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option.
    ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 313, in print_wrapper_func
        result = func(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 386, in inverted_operation_enable_disable_wrapper_func
        result = func(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 56, in get_replacement_parameter_wrapper_func
        func(*args, **kwargs)
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Conv.py", line 459, in make_node
        conv_bias(
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Conv.py", line 308, in conv_bias
        tf.nn.convolution(
      File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
        raise e.with_traceback(filtered_tb) from None
      File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/layers/core/tf_op_layer.py", line 119, in handle
        return TFOpLambda(op)(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/utils/traceback_utils.py", line 72, in error_handler
        del filtered_tb
    ValueError: Exception encountered when calling layer "tf.nn.convolution_35" (type TFOpLambda).
    
    Depth of input (384) is not a multiple of input depth of filter (256) for '{{node tf.nn.convolution_35/convolution}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](Placeholder, tf.nn.convolution_35/convolution/filter)' with input shapes: [1,64,160,384], [1,1,256,256].
    
    Call arguments received by layer "tf.nn.convolution_35" (type TFOpLambda):
      • input=tf.Tensor(shape=(1, 64, 160, 384), dtype=float32)
      • filters=tf.Tensor(shape=(1, 1, 256, 256), dtype=float32)
      • strides=['1', '1']
      • padding='SAME'
      • data_format=None
      • dilations=['1', '1']
      • name=None
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/dist-packages/IPython/core/interactiveshell.py", line 3553, in run_code
        exec(code_obj, self.user_global_ns, self.user_ns)
      File "<ipython-input-2-960d66694119>", line 5, in <cell line: 0>
        model.export(format="tfjs")
      File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/model.py", line 730, in export
        return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/exporter.py", line 468, in __call__
        f[5], keras_model = self.export_saved_model()
                            ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/exporter.py", line 192, in outer_func
        f, model = inner_func(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/exporter.py", line 975, in export_saved_model
        keras_model = onnx2tf.convert(
                      ^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/onnx2tf.py", line 1149, in convert
        op.make_node(
      File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 379, in print_wrapper_func
        sys.exit(1)
    SystemExit: 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py", line 1101, in get_records
        return _fixed_getinnerframes(etb, number_of_lines_of_context, tb_offset)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py", line 248, in wrapped
        return f(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py", line 281, in _fixed_getinnerframes
        records = fix_frame_records_filenames(inspect.getinnerframes(etb, context))
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/inspect.py", line 1739, in getinnerframes
        traceback_info = getframeinfo(tb, context)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/inspect.py", line 1671, in getframeinfo
        lineno = frame.f_lineno
                 ^^^^^^^^^^^^^^
    AttributeError: 'tuple' object has no attribute 'f_lineno'
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    [/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py](https://localhost:8080/#) in print_wrapper_func(*args, **kwargs)
        312         try:
    --> 313             result = func(*args, **kwargs)
        314 
    
    21 frames
    ValueError: Exception encountered when calling layer "tf.nn.convolution_35" (type TFOpLambda).
    
    Depth of input (384) is not a multiple of input depth of filter (256) for '{{node tf.nn.convolution_35/convolution}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](Placeholder, tf.nn.convolution_35/convolution/filter)' with input shapes: [1,64,160,384], [1,1,256,256].
    
    Call arguments received by layer "tf.nn.convolution_35" (type TFOpLambda):
      • input=tf.Tensor(shape=(1, 64, 160, 384), dtype=float32)
      • filters=tf.Tensor(shape=(1, 1, 256, 256), dtype=float32)
      • strides=['1', '1']
      • padding='SAME'
      • data_format=None
      • dilations=['1', '1']
      • name=None
    
    During handling of the above exception, another exception occurred:
    
    SystemExit                                Traceback (most recent call last)
        [... skipping hidden 1 frame]
    
    SystemExit: 1
    
    During handling of the above exception, another exception occurred:
    
    TypeError                                 Traceback (most recent call last)
        [... skipping hidden 1 frame]
    
    [/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py](https://localhost:8080/#) in find_recursion(etype, value, records)
        380     # first frame (from in to out) that looks different.
        381     if not is_recursion_error(etype, value, records):
    --> 382         return len(records), 0
        383 
        384     # Select filename, lineno, func_name to track frames with
    
    TypeError: object of type 'NoneType' has no len()
  3. WereUs666GitHub

    Hello, I tried to convert the yolo pose model into tfjs format. Then I encountered an issue. I am using AWS EC2 Linux server to run the code.

    python version: 3.12.0 Pytorch version: 2.6.0+cu124 tensorflow version: 2.18.0

    This is the code:

    from ultralytics import YOLO
    model = YOLO("yolo11n-pose")
    model.export(format="tfjs")

    This is the error:

    Traceback (most recent call last):
      File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 312, in print_wrapper_func
        result = func(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 385, in inverted_operation_enable_disable_wrapper_func
        result = func(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 55, in get_replacement_parameter_wrapper_func
        func(*args, **kwargs)
      File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/ops/Add.py", line 281, in make_node
        merge_two_consecutive_identical_ops_into_one(
      File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 5466, in merge_two_consecutive_identical_ops_into_one
        tf.math.add(
      File "/opt/conda/envs/py312/lib/python3.12/site-packages/tensorflow/python/ops/weak_tensor_ops.py", line 142, in wrapper
        return op(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/envs/py312/lib/python3.12/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
        raise e.with_traceback(filtered_tb) from None
      File "/opt/conda/envs/py312/lib/python3.12/site-packages/tf_keras/src/layers/core/tf_op_layer.py", line 119, in handle
        return TFOpLambda(op)(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/envs/py312/lib/python3.12/site-packages/tf_keras/src/utils/traceback_utils.py", line 72, in error_handler
        del filtered_tb
            ^^^^^^^^^^^
    ValueError: Exception encountered when calling layer "tf.math.add_5" (type TFOpLambda).
    
    Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_5/Add}} = AddV2[T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,16,160,32], [1,16,160,16].
    
    Call arguments received by layer "tf.math.add_5" (type TFOpLambda):
      • x=tf.Tensor(shape=(1, 16, 160, 32), dtype=float32)
      • y=tf.Tensor(shape=(1, 16, 160, 16), dtype=float32)
      • name='/model.2/m.0/Add'
    
    ERROR: input_onnx_file_path: yolo11n-pose.onnx
    ERROR: onnx_op_name: /model.2/m.0/Add
    ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement
    ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again.
    ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option.
    ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.
    1. Glenn JocherUltralytics(수정됨)

      @WereUs666 thanks for sharing the details. For pose models in TF.js format, we recommend trying model.export(format='tfjs', imgsz=(640, 640)) with a fixed input size. Note that tfjs is deprecated since Ultralytics 8.4.83 and now exports a LiteRT model, so for new work use format='litert' (see the LiteRT guide). If issues persist, please ensure you're using the latest ultralytics and onnx2tf packages. For commercial deployment needs, visit https://ultralytics.com/license.

  4. priyalpatel1503GitHub

    I am working on an Angular project, and I have integrated the YOLOv8 model using TensorFlow.js. However, I am facing an issue where the model does not provide the predicted bounding box coordinates, but instead, I receive the following tensor output: { "kept": false, "isDisposedInternal": false, "shape": [1, 84, 8400], "dtype": "float32", "size": 705600, "strides": [705600, 8400], "dataId": {"id": 801}, "id": 492, "rankType": "3", "scopeId": 7 } This output is not giving me the expected bounding box coordinates. I am unsure how to extract the bounding boxes from this tensor output and interpret the predictions. Could you please guide me on how to retrieve the bounding box coordinates, class labels, and confidence scores from this tensor?

    Here is the code I am using to load and run the model:

    // Loading YOLOv8 model this.model = await tf.loadGraphModel('../../assets/yolov8n_web_model/model.json'); alert('YOLOv8 model loaded successfully');

    // Preparing the image for model input const image = document.getElementById('input-image') as HTMLImageElement; const tensor = tf.browser .fromPixels(image) .resizeBilinear([640, 640]) // Resize to model input size .toFloat() .expandDims(0) .div(tf.scalar(255.0)); // Normalize

    // Running the model and obtaining predictions const predictions = await this.model.execute(tensor); console.log('Predictions:', predictions); To clarify, I am working with the model directly in Angular using TensorFlow.js and do not have a Python backend. Any advice on how to proceed in this setup would be greatly appreciated.

    Thank you!

    1. Glenn JocherUltralytics

      @priyalpatel1503 the tensor output you are encountering contains raw predictions, which need decoding to extract bounding box coordinates, class labels, and confidence scores. YOLO models typically output a tensor where each grid cell holds predictions including bounding boxes, objectness scores, and class probabilities. You can apply non-maximum suppression (NMS) and appropriate post-processing on this tensor to interpret the results. Refer to the TensorFlow.js YOLO post-processing example for guidance, as it follows a similar concept.

    2. priyalpatel1503GitHub

      I have tried implementing your above solution to run inference on an image, but I am encountering an issue where the predictions object is not iterable. Despite following the suggested solutions, it still doesn't work.

      Here's the solution I tried: async function runInference() { // Load the model const model = await tf.loadGraphModel('/path/to/yolov8n_web_model/model.json');

      // Load and preprocess the image
      const img = document.getElementById('input-image');
      const tensor = tf.browser.fromPixels(img).expandDims(0).toFloat().div(tf.scalar(255));
      
      // Run inference
      const predictions = await model.executeAsync(tensor);
      
      // Process the output tensors
      const [boxes, scores, classes] = predictions;
      const boxesData = boxes.dataSync();
      const scoresData = scores.dataSync();
      const classesData = classes.dataSync();
      
      // Filter predictions by confidence threshold
      const confidenceThreshold = 0.5;
      const validDetections = [];
      for (let i = 0; i < scoresData.length; i++) {
          if (scoresData[i] > confidenceThreshold) {
              validDetections.push({
                  box: boxesData.slice(i * 4, (i + 1) * 4),
                  score: scoresData[i],
                  class: classesData[i]
              });
          }
      }
      
      // Display results
      validDetections.forEach(detection => {
          const [x, y, width, height] = detection.box;
          const score = detection.score;
          const className = detection.class;
      
          // Draw bounding box and label on the image
          // (Implement your drawing logic here)
      });

      }

      window.onload = runInference; However, when running this code, I am getting the following error:

      Error: predictions is not iterable

      I have tried several approaches based on your earlier suggestions, but none of them resolved the issue. I am unsure how to correctly extract the bounding boxes, class labels, and confidence scores from the output.

      Could you please provide guidance on how to properly handle the output of model.executeAsync(tensor) and how to extract the predictions (bounding boxes, class labels, and scores)?

    3. Glenn JocherUltralytics

      The error occurs because model.executeAsync(tensor) returns a single tensor or an array of tensors, depending on the model's output structure. In your case, ensure the model's output aligns with your extraction logic. Use predictions.array() or predictions.dataSync() to inspect the raw output structure and adjust your code to handle it correctly. For YOLO models, you may need to decode the tensor output into bounding boxes, classes, and scores using custom logic or post-processing functions.

    4. priyalpatel1503GitHub

      @glenn-jocher After calling model.executeAsync(tensor), the output I receive is a tensor object with the following shape and details: { "kept": false, "isDisposedInternal": false, "shape": [ 1, 84, 8400 ], "dtype": "float32", "size": 705600, "strides": [ 705600, 8400 ], "dataId": { "id": 864 }, "id": 555, "rankType": "3", "scopeId": 641 }

      1. When using predictions.array(), it returns a Promise in TensorFlow.js .The Promise resolves to a nested array structure: [ [Array(8400), Array(8400), ..., Array(8400)] ] (84 arrays, each containing 8400 values).
      2. When using predictions.dataSync(), I receive a Float32Array with 705600 values.

      I need guidance on how to properly decode and interpret this output, especially in terms of extracting bounding box coordinates, class IDs, and class probabilities. Could you provide advice or code examples for decoding these predictions, particularly for the YOLOv8 model in TensorFlow.js? Any references, examples, or explanations will be helpful.

    5. Glenn JocherUltralytics(수정됨)

      To decode the YOLOv8 TensorFlow.js output, you need to understand that the tensor contains raw predictions. The shape [1, 84, 8400] represents [batch_size, num_features, num_predictions_per_feature]. Here's a breakdown of processing it:

      1. Use predictions.array() to resolve the tensor into a nested array structure.
      2. The 84 features are the bounding box (x, y, w, h) followed by 80 class scores. YOLOv8 has no separate objectness score:
        • The first 4 values are the bounding box coordinates.
        • The remaining 80 values (indices 4–83) are the per-class scores.
      3. For each prediction:
        • Scale the bounding box coordinates to your image size if they are normalized.
        • Take the highest class score as the confidence, filter by a confidence threshold, and apply Non-Maximum Suppression (NMS) to remove duplicates.

      For detailed implementation, you can refer to TensorFlow.js's post-processing examples or adapt the logic to YOLOv8's output format. Let us know if you need further clarification!

  5. lilineboGitHub

    I'm having performance issues when dealing with tensors. After the inference is completed, nms needs to be performed, and then I perform the following operation const indicesTensor = await tf.whereAsync(xcSlice); which takes about 2s, which is too long.xcSlice is [21504].In my expectation, this will only take up to 400 milliseconds

    1. Glenn JocherUltralytics

      @mozeqiu123 the delay you're encountering with tf.whereAsync may be due to the large size of your tensor. To improve performance, consider optimizing the data preparation and operations preceding this step. Also, ensure you're leveraging appropriate TensorFlow.js backends like WebGL or WASM for better execution speed.

    2. lilineboGitHub

      Yes, my tensor is very large. What exactly do you mean by optimizing data preparation and operations?

    3. Glenn JocherUltralytics

      Optimizing data preparation and operations can involve ensuring the tensor is only as large as necessary by filtering or reducing dimensions early, leveraging TensorFlow.js efficient backends (e.g., WebGL for GPU acceleration), and minimizing unnecessary tensor transformations before calling costly operations. You can also experiment with batching or slicing your data into smaller chunks before processing.

    4. lilineboGitHub

      Thanks for the reply. I turned on webgl at the beginning, but it was still very slow. Then I solved the problem by changing to a smaller model.

    5. Glenn JocherUltralytics

      Thank you for sharing your update! Switching to a smaller model is a great solution for handling performance constraints. If further optimizations are needed, consider exporting your YOLO11 to TensorFlow.js or experimenting with different backends like WebGL or WASM. For more context, you can refer to the TF.js documentation.

  6. viniciusmraGitHub

    I've been trying to export some models that I trained, but I keep getting several errors when trying to run the command. Erros:

    requirements: Ultralytics requirements ['sng4onnx>=1.0.1', 'onnx_graphsurgeon>=0.3.26', 'onnxslim>=0.1.31', 'tflite_support', 'onnxruntime'] not found, attempting AutoUpdate...

    The message says that the Ultralytics library is missing these dependencies (which is strange because I just installed Ultralytics, so it should be complete). The 'autoupdate' fails, and installing them manually also fails..

    1. Glenn JocherUltralytics

      @viniciusmra the error indicates that some dependencies required for exporting are missing. First, try updating all packages by running pip install ultralytics --upgrade to ensure you have the latest version. If manual dependency installation issues persist, verify your Python environment and ensure compatibility with the supported Ultralytics and dependency versions. For further guidance, refer to our Common Issues guide. Let me know if you encounter further issues.

      1
  7. lilineboGitHub

    Hello author, I am a newbie. I trained a yolov11n OBB detection model and then converted it to tfjs format. I found that the pt model inference result is an object, so how to parse the result in tfjs?

    1. Glenn JocherUltralytics

      @mozeqiu123 to parse the inference results in TensorFlow.js, check the format of the output from the exported YOLO model. Typically, it should provide arrays of tensor data. Map the output tensors to expected object properties like bounding box coordinates, class IDs, and confidence scores in your application logic to process and display the results. Let me know if you need more details!

      1
    2. Glenn JocherUltralytics

      Thank you for sharing the reference issue. To parse YOLO11 OBB detection results in TensorFlow.js, you will need to interpret the model's output tensors correctly based on your trained model's format. Typically, the outputs include bounding box coordinates, class predictions, confidence scores, and oriented box angles. You can check the model's documentation or inspect the export code to map these outputs properly. Let us know if further clarification is needed!

  8. venkatshan707GitHub(수정됨)

    @glenn-jocher sorry for correcting. When I convert yolov8n.pt I am getting 4 bin files. when I convert yolov8m.pt and yolov8m-obb.pt I am getting 26 bin files. I could understand your point explains model complexity of yolov8m.pt and yolov8m-obb.pt are higher than yolov8n.pt. I have tried finding code snippet which can help us pruning the model and gives us small number bin files. But I have failed in that.

    Would you kindly help me with code example which can fix this issue ?

    #This is the code I have used to get web model from official yolov8m-obb.pt

    from ultralytics import YOLO

    Load the YOLO11 model

    model = YOLO("yolov8m-obb.pt")

    Export the model to TF.js format

    model.export(format="tfjs") # creates '/yolov8m-obb.pt_web_model'

    more_bins

    1. Glenn JocherUltralytics

      @venkatshan707 to reduce the number of bin files when exporting to TF.js, consider model pruning or quantization techniques. Unfortunately, we don't provide specific code examples for this, but you can explore TensorFlow.js documentation for optimization strategies. For more guidance, visit Ultralytics TF.js integration.

  9. Fernand0GCGitHub

    buenas, alguien podria ayudarme a interpretar mi output? el modelo es de deteccion de objetos y esta entrenado para reconocer 5 clases names: 0: black-shape 1: diana 2: targets-holes 3: targets-holes-2 4: targets-holes-black en la consola recibo el siguiente objeto: { "kept": false, "isDisposedInternal": false, "shape": [ 1, 9, 8400 ], "dtype": "float32", "size": 75600, "strides": [ 75600, 8400 ], "dataId": { "id": 801 }, "id": 492, "rankType": "3", "scopeId": 578 }

    pude acceder al tensor mediante predictions.array() lo cual me da un output de un arreglo de 9 arreglos. este arreglo contiene valores numericos que aun no logro decifrar. 0 : (8400) [2.8815157413482666, 16.799739837646484, 35.85880661010742 ...] 1 : (8400) [4.487062931060791, 6.242208003997803, 7.13232421875 ...] 2 : (8400) [6.577016353607178, 32.058311462402344, 74.6260757446289 ...] 3 : (8400) [8.778525352478027, 12.535199165344238, 13.99738883972168 ...] 4 : (8400) [0.000003607327244026237, 0.000010992955139954574, 0.000006420665613404708 ...] 5 : (8400) [0.0000034999770832655486, 0.000010910729542956688, 0.000004991020887246123 ...] 6 : (8400) [0.000057105469750240445, 0.000025020823159138672, 0.000024430628400295973 ...] 7 : (8400) [0.000027735275580198504, 0.000019603612599894404, 0.000015360352335846983 ...] 8 : (8400) [0.00006247103010537103, 0.000020798512196051888, 0.000007502545486204326 ...]

    me podrian ayudar a interpretar este output porfabor

    1. Glenn JocherUltralytics

      @Fernand0GC hola, para interpretar tu output, los valores numéricos en los arreglos representan las predicciones del modelo. Cada arreglo de 8400 valores corresponde a diferentes aspectos de la detección, como coordenadas de cajas delimitadoras y puntuaciones de confianza para cada clase. Te recomiendo revisar cómo se estructuran las salidas de tu modelo en la documentación de TensorFlow.js para entender mejor cada componente. Si necesitas más detalles sobre el formato TF.js, puedes visitar aquí.

    1. Glenn JocherUltralytics(수정됨)

      Update: the TF.js export is deprecated. As of Ultralytics 8.4.83, format="tfjs" exports a LiteRT .tflite model instead of a *_web_model/model.json; use format="litert" and see the LiteRT guide. The TF.js snippets below are legacy.

      Hello @teengineer,

      Thank you for reaching out and for your interest in exporting YOLOv8 models to TensorFlow.js! The process indeed allows for efficient, high-speed, and locally-run object detection inference on various platforms, including web browsers and Node.js environments.

      To get started, you can follow the example provided in our documentation to export your YOLOv8 model to the TF.js format. Here’s a concise example to guide you through the process:

      Python Example
      from ultralytics import YOLO
      
      # Load the YOLOv8 model
      model = YOLO("yolov8n.pt")
      
      # Export the model to TF.js format
      model.export(format="tfjs")  # creates '/yolov8n_web_model'
      
      # Load the exported TF.js model
      tfjs_model = YOLO("./yolov8n_web_model")
      
      # Run inference
      results = tfjs_model("https://ultralytics.com/images/bus.jpg")
      CLI Example
      # Export a YOLOv8n PyTorch model to TF.js format
      yolo export model=yolov8n.pt format=tfjs  # creates '/yolov8n_web_model'
      
      # Run inference with the exported model
      yolo predict model='./yolov8n_web_model' source='https://ultralytics.com/images/bus.jpg'

      For more detailed instructions, you can visit our TF.js integration guide.

      If you encounter any issues, please ensure you are using the latest version of the Ultralytics package. Additionally, providing a minimum reproducible example of your issue can greatly help in diagnosing and resolving any problems. You can find more information on creating a reproducible example here.

      Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊

    2. Glenn JocherUltralytics(수정됨)

      Update: the TF.js export is deprecated. As of Ultralytics 8.4.83, format="tfjs" exports a LiteRT .tflite model instead of a *_web_model/model.json; use format="litert" and see the LiteRT guide. The TF.js snippets below are legacy.

      Hi @salim4n,

      Thank you for your message! I'm glad to hear you're working on integrating YOLOv8 with a React Chrome extension. To help you better understand the predictions and bounding boxes, you might want to inspect the structure of the results returned by the model.

      Here’s a quick example to guide you on how to handle the predictions in your React app:

      Example Code Snippet
      import React, { useEffect, useState } from 'react';
      import { YOLO } from 'ultralytics';
      
      const App = () => {
        const [results, setResults] = useState(null);
      
        useEffect(() => {
          const runInference = async () => {
            const model = await YOLO.load('./yolov8n_web_model');
            const results = await model.predict('https://ultralytics.com/images/bus.jpg');
            setResults(results);
          };
      
          runInference();
        }, []);
      
        return (
          <div>
            {results && results.map((result, index) => (
              <div key={index}>
                <h3>Prediction {index + 1}</h3>
                <p>Class: {result.class}</p>
                <p>Confidence: {result.confidence}</p>
                <p>Bounding Box: {result.bbox.join(', ')}</p>
              </div>
            ))}
          </div>
        );
      };
      
      export default App;
      Explanation
      1. Loading the Model: The YOLO.load function loads the exported TF.js model.
      2. Running Inference: The model.predict function runs inference on the provided image URL.
      3. Displaying Results: The results are stored in the state and displayed in the component.

      This should help you visualize the predictions and bounding boxes in your React app. For more detailed information, you can refer to our TF.js integration guide.

      If you encounter any issues, please ensure you are using the latest version of the Ultralytics package. Additionally, providing a minimum reproducible example can greatly help in diagnosing and resolving any problems. You can find more information on creating a reproducible example here.

      Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊

    3. salim4nGitHub

      Hi @glenn-jocher , thank's for your response, but I don't find any package ultralytics on NPM, with tfjs the load-graph on custom model is very hard to use. I need to try more on a server side version, trying implementation directly on a chrome extension is not the easy way.

    4. Glenn JocherUltralytics(수정됨)

      Update: the TF.js export is deprecated. As of Ultralytics 8.4.83, format="tfjs" exports a LiteRT .tflite model instead of a *_web_model/model.json; use format="litert" and see the LiteRT guide. The TF.js snippets below are legacy.

      Hi @salim4n,

      Thank you for your feedback and for sharing your experience. I understand that integrating TensorFlow.js (TF.js) models directly into a Chrome extension can be challenging, especially when dealing with custom models.

      To address your concerns:

      1. NPM Package: Currently, there is no ultralytics package available on NPM. The recommended approach is to use the Python package to export your model to the TF.js format and then integrate the exported model into your JavaScript project.

      2. Loading Custom Models in TF.js: Loading custom models in TF.js can indeed be complex. Here’s a simplified approach to help you get started:

        • Export the Model: Use the Ultralytics Python package to export your YOLOv8 model to the TF.js format.
        • Integrate with TF.js: Use the exported model in your JavaScript code.
      Example Code for Integration

      Here’s a basic example to load and use the exported TF.js model in a Chrome extension:

      Export the Model (Python)
      from ultralytics import YOLO
      
      # Load the YOLOv8 model
      model = YOLO("yolov8n.pt")
      
      # Export the model to TF.js format
      model.export(format="tfjs")  # creates '/yolov8n_web_model'
      Load and Use the Model (JavaScript)
      import * as tf from '@tensorflow/tfjs';
      
      async function loadModel() {
        const model = await tf.loadGraphModel('/path/to/yolov8n_web_model/model.json');
        return model;
      }
      
      async function runInference(imageElement) {
        const model = await loadModel();
        const tensor = tf.browser.fromPixels(imageElement).expandDims(0);
        const predictions = await model.executeAsync(tensor);
        console.log(predictions);
      }
      
      // Example usage
      const imageElement = document.getElementById('input-image');
      runInference(imageElement);
      Additional Resources

      For more detailed instructions on deploying TF.js models, you can refer to:

      If you encounter any issues, please ensure you are using the latest version of the Ultralytics package and provide a minimum reproducible example if possible. This helps us diagnose and resolve any problems more effectively. You can find more information on creating a reproducible example here.

      Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊

    5. wuhongleiGitHub

      i have the same problem about how to get the boxes, scores and classes from the predictions variable, predictions is an object ,not an array, there is no property named boxes, scores and classes, does anyone know how to solve this, very appreciate 😄

      my html code is below:

      <!DOCTYPE html>
      <html>
        <head>
          <title>YOLOv8 Model Inference</title>
        </head>
        <body>
          <!-- 用于显示输入图片和绘制推理结果 -->
          <canvas id="input-image" width="640" height="640"></canvas>
          <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
          <script>
            async function runModel() {
              const model = await tf.loadGraphModel("./best_web_model/model.json");
      
              // 1. 准备图片数据(这里使用示例图片)
              const imgElement = document.getElementById("input-image");
              const tensor = tf.browser
                .fromPixels(imgElement)
                .expandDims(0)
                .toFloat()
                .div(tf.scalar(255));
      
              // 2. 进行推理
              const predictions = await model.executeAsync([tensor]);
            }
          </script>
          <script>
            // 获取 canvas 元素
            const canvas = document.getElementById("input-image");
            const ctx = canvas.getContext("2d");
      
            // 创建新的图片对象
            const img = new Image();
      
            // 图片加载完成后绘制到 canvas 上
            img.onload = function () {
              ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
              runModel();
            };
      
            // 设置图片的来源
            img.src = "./imgs/20217191442296037.jpg"; // 替换为您的图片路径
          </script>
        </body>
      </html>

      the output of predictions is below:

      {
          "kept": false,
          "isDisposedInternal": false,
          "shape": [
              1,
              5,
              8400
          ],
          "dtype": "float32",
          "size": 42000,
          "strides": [
              42000,
              8400
          ],
          "dataId": {
              "id": 1212
          },
          "id": 701,
          "rankType": "3",
          "scopeId": 879
      }

      predictions

    6. Glenn JocherUltralytics(수정됨)

      Update: the TF.js export is deprecated. As of Ultralytics 8.4.83, format="tfjs" exports a LiteRT .tflite model instead of a *_web_model/model.json; use format="litert" and see the LiteRT guide. The TF.js snippets below are legacy.

      Hi @wuhonglei,

      Thank you for your detailed question and for sharing your code! 😊

      To extract the boxes, scores, and classes from the predictions object in TensorFlow.js, you need to process the tensor outputs correctly. The predictions object you have is a tensor, and you can use TensorFlow.js methods to extract the necessary information.

      Here’s how you can modify your code to get the boxes, scores, and classes:

      Updated JavaScript Code
      import * as tf from '@tensorflow/tfjs';
      
      async function runModel() {
        const model = await tf.loadGraphModel('./best_web_model/model.json');
      
        // Prepare image data
        const imgElement = document.getElementById('input-image');
        const tensor = tf.browser.fromPixels(imgElement).expandDims(0).toFloat().div(tf.scalar(255));
      
        // Run inference
        const predictions = await model.executeAsync(tensor);
      
        // Extract boxes, scores, and classes
        const [boxes, scores, classes] = await Promise.all([
          predictions[0].array(),
          predictions[1].array(),
          predictions[2].array()
        ]);
      
        console.log('Boxes:', boxes);
        console.log('Scores:', scores);
        console.log('Classes:', classes);
      
        // Optionally, draw boxes on the canvas
        drawBoxes(imgElement, boxes, scores, classes);
      }
      
      function drawBoxes(imgElement, boxes, scores, classes) {
        const canvas = document.getElementById('input-image');
        const ctx = canvas.getContext('2d');
        ctx.clearRect(0, 0, canvas.width, canvas.height);
        ctx.drawImage(imgElement, 0, 0, canvas.width, canvas.height);
      
        boxes.forEach((box, i) => {
          const [y, x, height, width] = box;
          ctx.strokeStyle = 'red';
          ctx.lineWidth = 2;
          ctx.strokeRect(x, y, width, height);
          ctx.fillStyle = 'red';
          ctx.fillText(`Class: ${classes[i]}, Score: ${scores[i].toFixed(2)}`, x, y > 10 ? y - 5 : 10);
        });
      }
      
      // Load image and run model
      const img = new Image();
      img.onload = function () {
        const canvas = document.getElementById('input-image');
        const ctx = canvas.getContext('2d');
        ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
        runModel();
      };
      img.src = './imgs/20217191442296037.jpg'; // Replace with your image path
      Explanation
      1. Model Loading: Load the model using tf.loadGraphModel.
      2. Image Preparation: Convert the image to a tensor and normalize it.
      3. Run Inference: Execute the model and get the predictions.
      4. Extract Data: Use array() to convert tensors to JavaScript arrays.
      5. Draw Boxes: Optionally, draw the bounding boxes on the canvas.

      This should help you extract and visualize the boxes, scores, and classes from your predictions. If you encounter any issues, please ensure you are using the latest version of TensorFlow.js and the Ultralytics package.

      For more detailed information, you can refer to our TF.js integration guide.

      Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊

    7. wuhongleiGitHub

      thanks very much for your detailed response, In order to better assist future developers, I will paste the code that I have successfully run in the end。 My business background: i use the yolov8 to detect the penguin in the image(object detection ). my final html code:

      <!DOCTYPE html>
      <html>
        <head>
          <title>YOLOv8 Object Detection with TensorFlow.js</title>
          <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
        </head>
        <body>
          <input type="file" id="upload" accept="image/*" />
          <canvas id="canvas"></canvas>
          <script>
            async function loadModel() {
              // 加载模型
              const model = await tf.loadGraphModel("./best_web_model/model.json");
              return model;
            }
      
            async function predict(model, imageElement) {
              // 将图像转换为张量
              const tensor = tf.browser
                .fromPixels(imageElement)
                .resizeNearestNeighbor([640, 640]) // 调整大小到模型输入大小
                .toFloat()
                .expandDims(0)
                .div(tf.scalar(255.0));
      
              // inference
              const predictions = await model.executeAsync(tensor);
      
              //  predictions shape is [1, 5, 8400]
              const reshapedPredictions = predictions.reshape([5, 8400]);
              const boxesTensor = reshapedPredictions.slice([0, 0], [4, 8400]);
              const scoresTensor = reshapedPredictions
                .slice([4, 0], [1, 8400])
                .squeeze();
      
              // Assuming that the category label of each box is fixed (you can adjust it as needed)
              const classesTensor = tf.fill([8400], 0);
      
              return {
                boxes: boxesTensor,
                scores: scoresTensor,
                classes: classesTensor,
              };
            }
      
            function drawBoxes(imageElement, canvas, boxes, scores, classes) {
              const ctx = canvas.getContext("2d");
              canvas.width = imageElement.width;
              canvas.height = imageElement.height;
              ctx.drawImage(imageElement, 0, 0, canvas.width, canvas.height);
      
              const labels = ["企鹅"]; // Replace with actual label
              const validBoxes = [];
      
              for (let i = 0; i < scores.length; i++) {
                if (scores[i] > 0.5) {
                  const x_c = boxes[0][i] / 640;
                  const y_c = boxes[1][i] / 640;
                  const w = boxes[2][i] / 640;
                  const h = boxes[3][i] / 640;
      
                  const x = x_c - w / 2;
                  const y = y_c - h / 2;
      
                  validBoxes.push([x, y, w, h, scores[i], classes[i]]);
                }
              }
      
              validBoxes.forEach((box) => {
                const [x, y, w, h, score, className] = box;
                ctx.strokeStyle = "red";
                ctx.lineWidth = 2;
                ctx.strokeRect(
                  x * canvas.width,
                  y * canvas.height,
                  w * canvas.width,
                  h * canvas.height
                );
                const text = `${className}: ${(score * 100).toFixed(2)}%`;
                ctx.fillText(text, x * canvas.width, y * canvas.height - 10);
              });
            }
      
            document
              .getElementById("upload")
              .addEventListener("change", async (event) => {
                const file = event.target.files[0];
                const imageElement = new Image();
                const reader = new FileReader();
      
                reader.onload = () => {
                  imageElement.src = reader.result;
                  imageElement.onload = async () => {
                    const canvas = document.getElementById("canvas");
                    const model = await loadModel();
                    const { boxes, scores, classes } = await predict(
                      model,
                      imageElement
                    );
                    drawBoxes(
                      imageElement,
                      canvas,
                      boxes.arraySync(),
                      scores.arraySync(),
                      classes.arraySync()
                    );
                  };
                };
      
                reader.readAsDataURL(file);
              });
          </script>
        </body>
      </html>

      the predictions variable is a tensor, the format is as below:

      the chrome html running result is below:

    8. Glenn JocherUltralytics(수정됨)

      Update: the TF.js export is deprecated. As of Ultralytics 8.4.83, format="tfjs" exports a LiteRT .tflite model instead of a *_web_model/model.json; use format="litert" and see the LiteRT guide. The TF.js snippets below are legacy.

      Hi @wuhonglei,

      Thank you for sharing your detailed implementation and the final working code! It's fantastic to see how you've successfully integrated YOLOv8 with TensorFlow.js for penguin detection in your web application. 🐧

      For those looking to extract boxes, scores, and classes from the predictions tensor, your approach is spot on. Here's a concise summary and a few additional tips for clarity:

      Key Steps:
      1. Load the Model:

        const model = await tf.loadGraphModel("./best_web_model/model.json");
      2. Prepare the Image:

        const tensor = tf.browser.fromPixels(imageElement)
          .resizeNearestNeighbor([640, 640])
          .toFloat()
          .expandDims(0)
          .div(tf.scalar(255.0));
      3. Run Inference:

        const predictions = await model.executeAsync(tensor);
      4. Extract Predictions:

        const reshapedPredictions = predictions.reshape([5, 8400]);
        const boxesTensor = reshapedPredictions.slice([0, 0], [4, 8400]);
        const scoresTensor = reshapedPredictions.slice([4, 0], [1, 8400]).squeeze();
        const classesTensor = tf.fill([8400], 0); // Adjust as needed
      5. Draw Boxes:

        function drawBoxes(imageElement, canvas, boxes, scores, classes) {
          const ctx = canvas.getContext("2d");
          canvas.width = imageElement.width;
          canvas.height = imageElement.height;
          ctx.drawImage(imageElement, 0, 0, canvas.width, canvas.height);
        
          const labels = ["企鹅"]; // Replace with actual label
          const validBoxes = [];
        
          for (let i = 0; i < scores.length; i++) {
            if (scores[i] > 0.5) {
              const x_c = boxes[0][i] / 640;
              const y_c = boxes[1][i] / 640;
              const w = boxes[2][i] / 640;
              const h = boxes[3][i] / 640;
        
              const x = x_c - w / 2;
              const y = y_c - h / 2;
        
              validBoxes.push([x, y, w, h, scores[i], classes[i]]);
            }
          }
        
          validBoxes.forEach((box) => {
            const [x, y, w, h, score, className] = box;
            ctx.strokeStyle = "red";
            ctx.lineWidth = 2;
            ctx.strokeRect(
              x * canvas.width,
              y * canvas.height,
              w * canvas.width,
              h * canvas.height
            );
            const text = `${className}: ${(score * 100).toFixed(2)}%`;
            ctx.fillText(text, x * canvas.width, y * canvas.height - 10);
          });
        }
      Additional Tips:
      • Model Optimization: Ensure your model is optimized for TF.js to improve performance.
      • Threshold Adjustment: Adjust the score threshold (0.5 in this case) based on your application's requirements.
      • Class Labels: Replace the placeholder labels with actual class names if you have multiple classes.

      For more detailed information on deploying YOLOv8 models with TensorFlow.js, you can refer to our TF.js integration guide.

      If you encounter any issues or have further questions, feel free to reach out. We're here to help! 😊

      3