从 YOLO26 模型格式导出为 TF.js 模型格式(已弃用)
自 Ultralytics 8.4.83 起,tfjs(TensorFlow.js)导出格式已弃用,并由统一的 Google LiteRT 格式取代。LiteRT 可通过 LiteRT.js 在浏览器中运行(支持 WebGPU 加速),覆盖 TF.js 所支持的浏览器和 Node.js 用例,同时还支持移动设备、嵌入式设备和边缘设备,只需使用一种 .tflite 模型。
format="tfjs" 仍可使用,但会发出弃用警告,并改为导出 LiteRT 模型。今后请使用 format="litert";如需查看当前的导出说明和浏览器部署路径,请参阅 LiteRT 导出指南。
直接在浏览器或 Node.js 上部署机器学习模型可能比较棘手。你需要确保模型格式经过优化以提升运行速度,这样模型才能在用户设备上本地运行交互式应用。TensorFlow.js(也称为 TF.js)模型格式旨在以尽可能低的功耗实现快速运行。
为什么要导出为 TF.js?#
将机器学习模型导出为 TensorFlow.js 格式,可带来诸多优势,便于部署机器学习应用。TensorFlow.js 由 TensorFlow 团队开发,是更广泛 TensorFlow 生态系统的一部分。通过将敏感数据保留在设备上,它有助于增强用户隐私和安全性。下图展示了 TensorFlow.js 架构,以及机器学习模型如何转换并部署到 Web 浏览器和 Node.js 上。
在本地运行模型还能降低延迟,带来更灵敏的用户体验。TensorFlow.js 还支持离线运行,因此即使没有互联网连接,用户也能使用你的应用。TF.js 专为在资源有限的设备上高效运行复杂模型而设计,并且具有良好的可扩展性,支持 GPU 加速。
TF.js 的主要特性#
以下是让 TF.js 对开发者实用的主要特性:
-
跨平台支持:TensorFlow.js 可在浏览器和 Node.js 环境中使用,灵活适配不同平台的部署需求,让开发者更轻松地构建和部署应用。
-
支持多种后端:TensorFlow.js 支持多种计算后端,包括 CPU、用于 GPU 加速的 WebGL、可实现接近原生执行速度的 WebAssembly (WASM),以及支持高级浏览器机器学习功能的 WebGPU。
-
离线运行:借助 TensorFlow.js,模型无需互联网连接即可在浏览器中运行,因此可以开发可离线使用的应用。
使用 TensorFlow.js 的部署选项#
TF.js 提供了多种机器学习模型部署方式:
-
浏览器内的 ML 应用:你可以构建直接在浏览器中运行机器学习模型的 Web 应用。这样无需服务器端计算,还能减轻服务器负载。
-
Node.js 应用:TensorFlow.js 也支持在 Node.js 环境中部署,从而能够开发服务器端机器学习应用。这对于需要利用服务器计算能力或访问服务器端数据的应用尤其有用。
-
Chrome 扩展:一种有趣的部署场景是使用 TensorFlow.js 创建 Chrome 扩展。例如,你可以开发一个扩展,让用户在任意网页中的图像上单击鼠标右键,并使用预训练的 ML 模型对其进行分类。将 TensorFlow.js 集成到日常网页浏览体验中,可以根据机器学习结果即时提供见解或增强功能。
使用 LiteRT 替代 TF.js 导出#
对于新的导出,请将模型转换为 LiteRT。
安装#
要安装所需软件包,请运行:
# Install the required package for YOLO26
pip install ultralytics有关安装流程的详细说明和最佳实践,请查看我们的 Ultralytics 安装指南。安装 YOLO26 所需的软件包时,如果遇到困难,请查阅我们的常见问题指南,获取解决方案和建议。
用法#
所有 Ultralytics YOLO26 模型均支持开箱即用的导出,便于你将模型集成到首选的部署工作流中。你可以查看支持的导出格式和配置选项的完整列表,为你的应用选择最佳配置。
tfjs 参数现在会导出 LiteRT .tflite 模型,该模型支持在本地运行 导出、预测 和 验证 模式,并可通过 LiteRT.js 在浏览器中运行。
from ultralytics import YOLO
# 加载 YOLO26 模型
model = YOLO("yolo26n.pt")
# 将模型导出为 LiteRT 格式(取代 TF.js)
model.export(format="litert", imgsz=640) # 分类任务使用 imgsz=224导出的 LiteRT .tflite 模型可通过 yolo predict 和 yolo val 直接加载,并可通过 LiteRT.js 在浏览器中运行。
导出参数#
请参阅 LiteRT 导出参数。有关导出流程的更多详情,请访问 Ultralytics 导出文档页面。
部署导出的 YOLO26 TensorFlow.js 模型#
导出 YOLO26 模型后,下一步就是部署模型。导出会生成一个 LiteRT .tflite 模型,可在设备端运行,也可通过 LiteRT.js 在浏览器中运行。
以下是供参考的旧版 TensorFlow.js 运行时资源:
-
Chrome 扩展程序:介绍如何将 TF.js 模型部署到 Chrome 扩展程序的开发者文档。
-
在 Node.js 中运行 TensorFlow.js:介绍如何直接在 Node.js 中运行 TensorFlow.js 的 TensorFlow 指南。
-
在云平台上部署 TensorFlow.js Node 项目:介绍如何在云平台上部署 TensorFlow.js Node 项目的 TensorFlow 指南。
总结#
本指南保留了旧版 TF.js 部署相关内容。新导出请使用 LiteRT。
如需了解更多用法详情,请访问 TensorFlow.js 官方文档。
常见问题#
改为导出为 LiteRT。
format="tfjs"仍可使用,但会发出弃用警告,并导出相同的 LiteRT 模型:用法from ultralytics import YOLO # 加载 YOLO26 模型 model = YOLO("yolo26n.pt") # 将模型导出为 LiteRT 格式(取代 TF.js) model.export(format="litert", imgsz=640) # 分类任务使用 imgsz=224 # 导出的 '.tflite' 模型可通过 LiteRT.js 在浏览器中运行,也可以使用 yolo predict/val 在本地运行。如需了解更多受支持的导出选项,请访问 Ultralytics 关于部署选项的文档页面。
TensorFlow.js 专为在浏览器和 Node.js 环境中高效执行 ML 模型而设计。它能为基于浏览器的应用带来以下好处:
- 降低延迟:在本地运行机器学习模型,无需依赖服务器端计算即可立即获得结果。
- 提升隐私保护:将敏感数据保留在用户设备上,降低安全风险。
- 支持离线使用:模型无需互联网连接即可运行,确保功能稳定可用。
- 支持多种后端:提供多种后端选择,包括 CPU、WebGL、WebAssembly (WASM) 和 WebGPU,以满足不同的计算需求。
想进一步了解 TF.js?请查看 TensorFlow.js 官方指南。
评论45
提出问题、分享结果,或帮助其他人。
jupiter-mza
Hi, iam trying use yolov8n_web_model in my REACT project, all works fine, the model was loaded, predictions works, but the result, even making conversions stay wrong, its returned from my predictions: { "kept": false, "isDisposedInternal": false, "shape": [ 1, 84, 8400 ], "dtype": "float32", "size": 705600, "strides": [ 705600, 8400 ], "dataId": { "id": 1030 }, "id": 719, "rankType": "3", "scopeId": 575 } when i try to catch the results using this: const handleIdentify = async () => { if (!model) { console.warn('Modelo não carregado.'); return; } const canvas = canvasRef.current; const imageElement = canvas; const tensor = tf.browser.fromPixels(imageElement).expandDims(0).toFloat().div(tf.scalar(255)); const predictions = await model.executeAsync(tensor); // console.log("Predictions:", predictions); const result = await predictions.array(); console.log("Result:", result); // // Parâmetros da sua imagem const imgWidth = 640;
const imgHeight = 480;// Array para armazenar deteções filtradas const detections = []; for (let i = 0; i < result[0].length; i++) { const prediction = result[0][i]; // vetor de 84 elementos const x = prediction[0]; const y = prediction[1]; const w = prediction[2]; const h = prediction[3]; const objConfidence = prediction[4]; const classProbs = prediction.slice(5); // de 5 a 83 // Encontra a classe com maior probabilidade const maxClassProb = Math.max(...classProbs); const classIndex = classProbs.indexOf(maxClassProb); // Calcula a confiança do objeto const confidence = objConfidence * maxClassProb; // Define um limiar de confiança const confidenceThreshold = 0.25; if (confidence > confidenceThreshold) { // Escala as coordenadas para o tamanho da imagem const bx = x * imgWidth; const by = y * imgHeight; const bw = w * imgWidth; const bh = h * imgHeight; detections.push({ bbox: [bx - bw / 2, by - bh / 2, bw, bh], confidence: confidence, class: classIndex, }); } console.log("Detections:", detections) };} its return a resultset completely wrong like this: [ { "bbox": [ -2065.0027465820312, -2358.991241455078, 12443.428955078125, 14857.036743164062 ], "confidence": 23974.878087386023, "class": 2154 }, { "bbox": [ 2059.2428588867188, 1746.5789794921875, 2604.3841552734375, 2417.2320556640625 ], "confidence": 2756.7185883242637, "class": 6330 }, { "bbox": [ 3684.2034912109375, 1313.7918090820312, 8792.0166015625, 16787.200927734375 ], "confidence": 18003.25029833475, "class": 8224 }, { "bbox": [ 4112.9534912109375, 3463.8272094726562, 5174.500732421875, 4886.737060546875 ], "confidence": 5542.858029322, "class": 8182 } ] why ?
SarvagyaVaish
What version of ultralytics was the tfjs export tested with?
I am running this minimal example to convert a model to tfjs.
from ultralytics import YOLO model = YOLO("yolo11l.pt") model.export(format='tfjs')Error i am seeing the following error:
Ultralytics 8.3.133 🚀 Python-3.11.12 torch-2.6.0+cu124 CPU (Intel Xeon 2.20GHz) YOLO11l summary (fused): 190 layers, 25,340,992 parameters, 0 gradients, 86.9 GFLOPs PyTorch: starting from 'yolo11l.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (49.0 MB) TensorFlow SavedModel: starting export with tensorflow 2.18.0... ONNX: starting export with onnx 1.18.0 opset 19... ONNX: slimming with onnxslim 0.1.52... ONNX: export success ✅ 4.4s, saved as 'yolo11l.onnx' (97.0 MB) TensorFlow SavedModel: starting TFLite export with onnx2tf 1.27.4... ERROR:root:Internal Python error in the inspect module. Below is the traceback from this internal error. ERROR: The trace log is below. Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 313, in print_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 386, in inverted_operation_enable_disable_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 56, in get_replacement_parameter_wrapper_func func(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Conv.py", line 459, in make_node conv_bias( File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Conv.py", line 308, in conv_bias tf.nn.convolution( File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/layers/core/tf_op_layer.py", line 119, in handle return TFOpLambda(op)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/utils/traceback_utils.py", line 72, in error_handler del filtered_tb ValueError: Exception encountered when calling layer "tf.nn.convolution_35" (type TFOpLambda). Depth of input (384) is not a multiple of input depth of filter (256) for '{{node tf.nn.convolution_35/convolution}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](Placeholder, tf.nn.convolution_35/convolution/filter)' with input shapes: [1,64,160,384], [1,1,256,256]. Call arguments received by layer "tf.nn.convolution_35" (type TFOpLambda): • input=tf.Tensor(shape=(1, 64, 160, 384), dtype=float32) • filters=tf.Tensor(shape=(1, 1, 256, 256), dtype=float32) • strides=['1', '1'] • padding='SAME' • data_format=None • dilations=['1', '1'] • name=None ERROR: input_onnx_file_path: yolo11l.onnx ERROR: onnx_op_name: wa/model.2/cv2/conv/Conv ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again. ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option. ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option. Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 313, in print_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 386, in inverted_operation_enable_disable_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 56, in get_replacement_parameter_wrapper_func func(*args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Conv.py", line 459, in make_node conv_bias( File "/usr/local/lib/python3.11/dist-packages/onnx2tf/ops/Conv.py", line 308, in conv_bias tf.nn.convolution( File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/layers/core/tf_op_layer.py", line 119, in handle return TFOpLambda(op)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/tf_keras/src/utils/traceback_utils.py", line 72, in error_handler del filtered_tb ValueError: Exception encountered when calling layer "tf.nn.convolution_35" (type TFOpLambda). Depth of input (384) is not a multiple of input depth of filter (256) for '{{node tf.nn.convolution_35/convolution}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](Placeholder, tf.nn.convolution_35/convolution/filter)' with input shapes: [1,64,160,384], [1,1,256,256]. Call arguments received by layer "tf.nn.convolution_35" (type TFOpLambda): • input=tf.Tensor(shape=(1, 64, 160, 384), dtype=float32) • filters=tf.Tensor(shape=(1, 1, 256, 256), dtype=float32) • strides=['1', '1'] • padding='SAME' • data_format=None • dilations=['1', '1'] • name=None During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/IPython/core/interactiveshell.py", line 3553, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "<ipython-input-2-960d66694119>", line 5, in <cell line: 0> model.export(format="tfjs") File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/model.py", line 730, in export return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/exporter.py", line 468, in __call__ f[5], keras_model = self.export_saved_model() ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/exporter.py", line 192, in outer_func f, model = inner_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/ultralytics/engine/exporter.py", line 975, in export_saved_model keras_model = onnx2tf.convert( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/onnx2tf/onnx2tf.py", line 1149, in convert op.make_node( File "/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py", line 379, in print_wrapper_func sys.exit(1) SystemExit: 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py", line 1101, in get_records return _fixed_getinnerframes(etb, number_of_lines_of_context, tb_offset) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py", line 248, in wrapped return f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py", line 281, in _fixed_getinnerframes records = fix_frame_records_filenames(inspect.getinnerframes(etb, context)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/inspect.py", line 1739, in getinnerframes traceback_info = getframeinfo(tb, context) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/inspect.py", line 1671, in getframeinfo lineno = frame.f_lineno ^^^^^^^^^^^^^^ AttributeError: 'tuple' object has no attribute 'f_lineno' --------------------------------------------------------------------------- ValueError Traceback (most recent call last) [/usr/local/lib/python3.11/dist-packages/onnx2tf/utils/common_functions.py](https://localhost:8080/#) in print_wrapper_func(*args, **kwargs) 312 try: --> 313 result = func(*args, **kwargs) 314 21 frames ValueError: Exception encountered when calling layer "tf.nn.convolution_35" (type TFOpLambda). Depth of input (384) is not a multiple of input depth of filter (256) for '{{node tf.nn.convolution_35/convolution}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](Placeholder, tf.nn.convolution_35/convolution/filter)' with input shapes: [1,64,160,384], [1,1,256,256]. Call arguments received by layer "tf.nn.convolution_35" (type TFOpLambda): • input=tf.Tensor(shape=(1, 64, 160, 384), dtype=float32) • filters=tf.Tensor(shape=(1, 1, 256, 256), dtype=float32) • strides=['1', '1'] • padding='SAME' • data_format=None • dilations=['1', '1'] • name=None During handling of the above exception, another exception occurred: SystemExit Traceback (most recent call last) [... skipping hidden 1 frame] SystemExit: 1 During handling of the above exception, another exception occurred: TypeError Traceback (most recent call last) [... skipping hidden 1 frame] [/usr/local/lib/python3.11/dist-packages/IPython/core/ultratb.py](https://localhost:8080/#) in find_recursion(etype, value, records) 380 # first frame (from in to out) that looks different. 381 if not is_recursion_error(etype, value, records): --> 382 return len(records), 0 383 384 # Select filename, lineno, func_name to track frames with TypeError: object of type 'NoneType' has no len()WereUs666
Hello, I tried to convert the yolo pose model into tfjs format. Then I encountered an issue. I am using AWS EC2 Linux server to run the code.
python version: 3.12.0 Pytorch version: 2.6.0+cu124 tensorflow version: 2.18.0
This is the code:
from ultralytics import YOLO model = YOLO("yolo11n-pose") model.export(format="tfjs")This is the error:
Traceback (most recent call last): File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 312, in print_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 385, in inverted_operation_enable_disable_wrapper_func result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 55, in get_replacement_parameter_wrapper_func func(*args, **kwargs) File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/ops/Add.py", line 281, in make_node merge_two_consecutive_identical_ops_into_one( File "/opt/conda/envs/py312/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 5466, in merge_two_consecutive_identical_ops_into_one tf.math.add( File "/opt/conda/envs/py312/lib/python3.12/site-packages/tensorflow/python/ops/weak_tensor_ops.py", line 142, in wrapper return op(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py312/lib/python3.12/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/opt/conda/envs/py312/lib/python3.12/site-packages/tf_keras/src/layers/core/tf_op_layer.py", line 119, in handle return TFOpLambda(op)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py312/lib/python3.12/site-packages/tf_keras/src/utils/traceback_utils.py", line 72, in error_handler del filtered_tb ^^^^^^^^^^^ ValueError: Exception encountered when calling layer "tf.math.add_5" (type TFOpLambda). Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_5/Add}} = AddV2[T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,16,160,32], [1,16,160,16]. Call arguments received by layer "tf.math.add_5" (type TFOpLambda): • x=tf.Tensor(shape=(1, 16, 160, 32), dtype=float32) • y=tf.Tensor(shape=(1, 16, 160, 16), dtype=float32) • name='/model.2/m.0/Add' ERROR: input_onnx_file_path: yolo11n-pose.onnx ERROR: onnx_op_name: /model.2/m.0/Add ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again. ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option. ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.
Glenn JocherUltralytics(已编辑)@WereUs666 thanks for sharing the details. For pose models in TF.js format, we recommend trying
model.export(format='tfjs', imgsz=(640, 640))with a fixed input size. Note thattfjsis deprecated since Ultralytics 8.4.83 and now exports a LiteRT model, so for new work useformat='litert'(see the LiteRT guide). If issues persist, please ensure you're using the latestultralyticsandonnx2tfpackages. For commercial deployment needs, visit https://ultralytics.com/license.
priyalpatel1503
I am working on an Angular project, and I have integrated the YOLOv8 model using TensorFlow.js. However, I am facing an issue where the model does not provide the predicted bounding box coordinates, but instead, I receive the following tensor output: { "kept": false, "isDisposedInternal": false, "shape": [1, 84, 8400], "dtype": "float32", "size": 705600, "strides": [705600, 8400], "dataId": {"id": 801}, "id": 492, "rankType": "3", "scopeId": 7 } This output is not giving me the expected bounding box coordinates. I am unsure how to extract the bounding boxes from this tensor output and interpret the predictions. Could you please guide me on how to retrieve the bounding box coordinates, class labels, and confidence scores from this tensor?
Here is the code I am using to load and run the model:
// Loading YOLOv8 model this.model = await tf.loadGraphModel('../../assets/yolov8n_web_model/model.json'); alert('YOLOv8 model loaded successfully');
// Preparing the image for model input const image = document.getElementById('input-image') as HTMLImageElement; const tensor = tf.browser .fromPixels(image) .resizeBilinear([640, 640]) // Resize to model input size .toFloat() .expandDims(0) .div(tf.scalar(255.0)); // Normalize
// Running the model and obtaining predictions const predictions = await this.model.execute(tensor); console.log('Predictions:', predictions); To clarify, I am working with the model directly in Angular using TensorFlow.js and do not have a Python backend. Any advice on how to proceed in this setup would be greatly appreciated.
Thank you!
Glenn JocherUltralytics@priyalpatel1503 the tensor output you are encountering contains raw predictions, which need decoding to extract bounding box coordinates, class labels, and confidence scores. YOLO models typically output a tensor where each grid cell holds predictions including bounding boxes, objectness scores, and class probabilities. You can apply non-maximum suppression (NMS) and appropriate post-processing on this tensor to interpret the results. Refer to the TensorFlow.js YOLO post-processing example for guidance, as it follows a similar concept.
priyalpatel1503
I have tried implementing your above solution to run inference on an image, but I am encountering an issue where the predictions object is not iterable. Despite following the suggested solutions, it still doesn't work.
Here's the solution I tried: async function runInference() { // Load the model const model = await tf.loadGraphModel('/path/to/yolov8n_web_model/model.json');
// Load and preprocess the image const img = document.getElementById('input-image'); const tensor = tf.browser.fromPixels(img).expandDims(0).toFloat().div(tf.scalar(255)); // Run inference const predictions = await model.executeAsync(tensor); // Process the output tensors const [boxes, scores, classes] = predictions; const boxesData = boxes.dataSync(); const scoresData = scores.dataSync(); const classesData = classes.dataSync(); // Filter predictions by confidence threshold const confidenceThreshold = 0.5; const validDetections = []; for (let i = 0; i < scoresData.length; i++) { if (scoresData[i] > confidenceThreshold) { validDetections.push({ box: boxesData.slice(i * 4, (i + 1) * 4), score: scoresData[i], class: classesData[i] }); } } // Display results validDetections.forEach(detection => { const [x, y, width, height] = detection.box; const score = detection.score; const className = detection.class; // Draw bounding box and label on the image // (Implement your drawing logic here) });}
window.onload = runInference; However, when running this code, I am getting the following error:
Error: predictions is not iterable
I have tried several approaches based on your earlier suggestions, but none of them resolved the issue. I am unsure how to correctly extract the bounding boxes, class labels, and confidence scores from the output.
Could you please provide guidance on how to properly handle the output of model.executeAsync(tensor) and how to extract the predictions (bounding boxes, class labels, and scores)?
Glenn JocherUltralyticsThe error occurs because
model.executeAsync(tensor)returns a single tensor or an array of tensors, depending on the model's output structure. In your case, ensure the model's output aligns with your extraction logic. Usepredictions.array()orpredictions.dataSync()to inspect the raw output structure and adjust your code to handle it correctly. For YOLO models, you may need to decode the tensor output into bounding boxes, classes, and scores using custom logic or post-processing functions.priyalpatel1503
@glenn-jocher After calling model.executeAsync(tensor), the output I receive is a tensor object with the following shape and details: { "kept": false, "isDisposedInternal": false, "shape": [ 1, 84, 8400 ], "dtype": "float32", "size": 705600, "strides": [ 705600, 8400 ], "dataId": { "id": 864 }, "id": 555, "rankType": "3", "scopeId": 641 }
- When using predictions.array(), it returns a Promise in TensorFlow.js .The Promise resolves to a nested array structure: [ [Array(8400), Array(8400), ..., Array(8400)] ] (84 arrays, each containing 8400 values).
- When using predictions.dataSync(), I receive a Float32Array with 705600 values.
I need guidance on how to properly decode and interpret this output, especially in terms of extracting bounding box coordinates, class IDs, and class probabilities. Could you provide advice or code examples for decoding these predictions, particularly for the YOLOv8 model in TensorFlow.js? Any references, examples, or explanations will be helpful.
Glenn JocherUltralytics(已编辑)To decode the YOLOv8 TensorFlow.js output, you need to understand that the tensor contains raw predictions. The shape
[1, 84, 8400]represents[batch_size, num_features, num_predictions_per_feature]. Here's a breakdown of processing it:- Use
predictions.array()to resolve the tensor into a nested array structure. - The 84 features are the bounding box (x, y, w, h) followed by 80 class scores. YOLOv8 has no separate objectness score:
- The first 4 values are the bounding box coordinates.
- The remaining 80 values (indices 4–83) are the per-class scores.
- For each prediction:
- Scale the bounding box coordinates to your image size if they are normalized.
- Take the highest class score as the confidence, filter by a confidence threshold, and apply Non-Maximum Suppression (NMS) to remove duplicates.
For detailed implementation, you can refer to TensorFlow.js's post-processing examples or adapt the logic to YOLOv8's output format. Let us know if you need further clarification!
- Use
lilinebo
I'm having performance issues when dealing with tensors. After the inference is completed, nms needs to be performed, and then I perform the following operation const indicesTensor = await tf.whereAsync(xcSlice); which takes about 2s, which is too long.xcSlice is [21504].In my expectation, this will only take up to 400 milliseconds
Glenn JocherUltralytics@mozeqiu123 the delay you're encountering with
tf.whereAsyncmay be due to the large size of your tensor. To improve performance, consider optimizing the data preparation and operations preceding this step. Also, ensure you're leveraging appropriate TensorFlow.js backends like WebGL or WASM for better execution speed.lilinebo
Yes, my tensor is very large. What exactly do you mean by optimizing data preparation and operations?
Glenn JocherUltralyticsOptimizing data preparation and operations can involve ensuring the tensor is only as large as necessary by filtering or reducing dimensions early, leveraging TensorFlow.js efficient backends (e.g., WebGL for GPU acceleration), and minimizing unnecessary tensor transformations before calling costly operations. You can also experiment with batching or slicing your data into smaller chunks before processing.
lilinebo
Thanks for the reply. I turned on webgl at the beginning, but it was still very slow. Then I solved the problem by changing to a smaller model.
Glenn JocherUltralyticsThank you for sharing your update! Switching to a smaller model is a great solution for handling performance constraints. If further optimizations are needed, consider exporting your YOLO11 to TensorFlow.js or experimenting with different backends like WebGL or WASM. For more context, you can refer to the TF.js documentation.
viniciusmra
I've been trying to export some models that I trained, but I keep getting several errors when trying to run the command. Erros:
requirements: Ultralytics requirements ['sng4onnx>=1.0.1', 'onnx_graphsurgeon>=0.3.26', 'onnxslim>=0.1.31', 'tflite_support', 'onnxruntime'] not found, attempting AutoUpdate...The message says that the Ultralytics library is missing these dependencies (which is strange because I just installed Ultralytics, so it should be complete). The 'autoupdate' fails, and installing them manually also fails..
Glenn JocherUltralytics@viniciusmra the error indicates that some dependencies required for exporting are missing. First, try updating all packages by running
pip install ultralytics --upgradeto ensure you have the latest version. If manual dependency installation issues persist, verify your Python environment and ensure compatibility with the supported Ultralytics and dependency versions. For further guidance, refer to our Common Issues guide. Let me know if you encounter further issues.1
lilinebo
Hello author, I am a newbie. I trained a yolov11n OBB detection model and then converted it to tfjs format. I found that the pt model inference result is an object, so how to parse the result in tfjs?
Glenn JocherUltralytics@mozeqiu123 to parse the inference results in TensorFlow.js, check the format of the output from the exported YOLO model. Typically, it should provide arrays of tensor data. Map the output tensors to expected object properties like bounding box coordinates, class IDs, and confidence scores in your application logic to process and display the results. Let me know if you need more details!
1lilinebo
Thank you for your reply, the specific information is here:https://github.com/ultralytics/ultralytics/issues/17768
Glenn JocherUltralyticsThank you for sharing the reference issue. To parse YOLO11 OBB detection results in TensorFlow.js, you will need to interpret the model's output tensors correctly based on your trained model's format. Typically, the outputs include bounding box coordinates, class predictions, confidence scores, and oriented box angles. You can check the model's documentation or inspect the export code to map these outputs properly. Let us know if further clarification is needed!
venkatshan707(已编辑)
@glenn-jocher sorry for correcting. When I convert yolov8n.pt I am getting 4 bin files. when I convert yolov8m.pt and yolov8m-obb.pt I am getting 26 bin files. I could understand your point explains model complexity of yolov8m.pt and yolov8m-obb.pt are higher than yolov8n.pt. I have tried finding code snippet which can help us pruning the model and gives us small number bin files. But I have failed in that.
Would you kindly help me with code example which can fix this issue ?
#This is the code I have used to get web model from official yolov8m-obb.pt
from ultralytics import YOLO
Load the YOLO11 model
model = YOLO("yolov8m-obb.pt")
Export the model to TF.js format
model.export(format="tfjs") # creates '/yolov8m-obb.pt_web_model'
Glenn JocherUltralytics@venkatshan707 to reduce the number of bin files when exporting to TF.js, consider model pruning or quantization techniques. Unfortunately, we don't provide specific code examples for this, but you can explore TensorFlow.js documentation for optimization strategies. For more guidance, visit Ultralytics TF.js integration.
Fernand0GC
buenas, alguien podria ayudarme a interpretar mi output? el modelo es de deteccion de objetos y esta entrenado para reconocer 5 clases names: 0: black-shape 1: diana 2: targets-holes 3: targets-holes-2 4: targets-holes-black en la consola recibo el siguiente objeto: { "kept": false, "isDisposedInternal": false, "shape": [ 1, 9, 8400 ], "dtype": "float32", "size": 75600, "strides": [ 75600, 8400 ], "dataId": { "id": 801 }, "id": 492, "rankType": "3", "scopeId": 578 }
pude acceder al tensor mediante predictions.array() lo cual me da un output de un arreglo de 9 arreglos. este arreglo contiene valores numericos que aun no logro decifrar. 0 : (8400) [2.8815157413482666, 16.799739837646484, 35.85880661010742 ...] 1 : (8400) [4.487062931060791, 6.242208003997803, 7.13232421875 ...] 2 : (8400) [6.577016353607178, 32.058311462402344, 74.6260757446289 ...] 3 : (8400) [8.778525352478027, 12.535199165344238, 13.99738883972168 ...] 4 : (8400) [0.000003607327244026237, 0.000010992955139954574, 0.000006420665613404708 ...] 5 : (8400) [0.0000034999770832655486, 0.000010910729542956688, 0.000004991020887246123 ...] 6 : (8400) [0.000057105469750240445, 0.000025020823159138672, 0.000024430628400295973 ...] 7 : (8400) [0.000027735275580198504, 0.000019603612599894404, 0.000015360352335846983 ...] 8 : (8400) [0.00006247103010537103, 0.000020798512196051888, 0.000007502545486204326 ...]
me podrian ayudar a interpretar este output porfabor
Glenn JocherUltralytics@Fernand0GC hola, para interpretar tu output, los valores numéricos en los arreglos representan las predicciones del modelo. Cada arreglo de 8400 valores corresponde a diferentes aspectos de la detección, como coordenadas de cajas delimitadoras y puntuaciones de confianza para cada clase. Te recomiendo revisar cómo se estructuran las salidas de tu modelo en la documentación de TensorFlow.js para entender mejor cada componente. Si necesitas más detalles sobre el formato TF.js, puedes visitar aquí.
teengineer
@glenn-jocher thank you for your time. This example is on your web site https://docs.ultralytics.com/integrations/tfjs/#usage
Glenn JocherUltralytics(已编辑)Update: the TF.js export is deprecated. As of Ultralytics 8.4.83,
format="tfjs"exports a LiteRT.tflitemodel instead of a*_web_model/model.json; useformat="litert"and see the LiteRT guide. The TF.js snippets below are legacy.Hello @teengineer,
Thank you for reaching out and for your interest in exporting YOLOv8 models to TensorFlow.js! The process indeed allows for efficient, high-speed, and locally-run object detection inference on various platforms, including web browsers and Node.js environments.
To get started, you can follow the example provided in our documentation to export your YOLOv8 model to the TF.js format. Here’s a concise example to guide you through the process:
Python Example
from ultralytics import YOLO # Load the YOLOv8 model model = YOLO("yolov8n.pt") # Export the model to TF.js format model.export(format="tfjs") # creates '/yolov8n_web_model' # Load the exported TF.js model tfjs_model = YOLO("./yolov8n_web_model") # Run inference results = tfjs_model("https://ultralytics.com/images/bus.jpg")CLI Example
# Export a YOLOv8n PyTorch model to TF.js format yolo export model=yolov8n.pt format=tfjs # creates '/yolov8n_web_model' # Run inference with the exported model yolo predict model='./yolov8n_web_model' source='https://ultralytics.com/images/bus.jpg'For more detailed instructions, you can visit our TF.js integration guide.
If you encounter any issues, please ensure you are using the latest version of the Ultralytics package. Additionally, providing a minimum reproducible example of your issue can greatly help in diagnosing and resolving any problems. You can find more information on creating a reproducible example here.
Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊
salim4n
Working well for me , I use in react chrome extension.
salim4n
sorry double post, but in fact i can't devine whos is pred, boxing, etc : https://github.com/salim4n/pretorian-ext/blob/main/src/App.tsx
Glenn JocherUltralytics(已编辑)Update: the TF.js export is deprecated. As of Ultralytics 8.4.83,
format="tfjs"exports a LiteRT.tflitemodel instead of a*_web_model/model.json; useformat="litert"and see the LiteRT guide. The TF.js snippets below are legacy.Hi @salim4n,
Thank you for your message! I'm glad to hear you're working on integrating YOLOv8 with a React Chrome extension. To help you better understand the predictions and bounding boxes, you might want to inspect the structure of the results returned by the model.
Here’s a quick example to guide you on how to handle the predictions in your React app:
Example Code Snippet
import React, { useEffect, useState } from 'react'; import { YOLO } from 'ultralytics'; const App = () => { const [results, setResults] = useState(null); useEffect(() => { const runInference = async () => { const model = await YOLO.load('./yolov8n_web_model'); const results = await model.predict('https://ultralytics.com/images/bus.jpg'); setResults(results); }; runInference(); }, []); return ( <div> {results && results.map((result, index) => ( <div key={index}> <h3>Prediction {index + 1}</h3> <p>Class: {result.class}</p> <p>Confidence: {result.confidence}</p> <p>Bounding Box: {result.bbox.join(', ')}</p> </div> ))} </div> ); }; export default App;Explanation
- Loading the Model: The
YOLO.loadfunction loads the exported TF.js model. - Running Inference: The
model.predictfunction runs inference on the provided image URL. - Displaying Results: The results are stored in the state and displayed in the component.
This should help you visualize the predictions and bounding boxes in your React app. For more detailed information, you can refer to our TF.js integration guide.
If you encounter any issues, please ensure you are using the latest version of the Ultralytics package. Additionally, providing a minimum reproducible example can greatly help in diagnosing and resolving any problems. You can find more information on creating a reproducible example here.
Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊
- Loading the Model: The
salim4n
Hi @glenn-jocher , thank's for your response, but I don't find any package ultralytics on NPM, with tfjs the load-graph on custom model is very hard to use. I need to try more on a server side version, trying implementation directly on a chrome extension is not the easy way.
Glenn JocherUltralytics(已编辑)Update: the TF.js export is deprecated. As of Ultralytics 8.4.83,
format="tfjs"exports a LiteRT.tflitemodel instead of a*_web_model/model.json; useformat="litert"and see the LiteRT guide. The TF.js snippets below are legacy.Hi @salim4n,
Thank you for your feedback and for sharing your experience. I understand that integrating TensorFlow.js (TF.js) models directly into a Chrome extension can be challenging, especially when dealing with custom models.
To address your concerns:
-
NPM Package: Currently, there is no
ultralyticspackage available on NPM. The recommended approach is to use the Python package to export your model to the TF.js format and then integrate the exported model into your JavaScript project. -
Loading Custom Models in TF.js: Loading custom models in TF.js can indeed be complex. Here’s a simplified approach to help you get started:
- Export the Model: Use the Ultralytics Python package to export your YOLOv8 model to the TF.js format.
- Integrate with TF.js: Use the exported model in your JavaScript code.
Example Code for Integration
Here’s a basic example to load and use the exported TF.js model in a Chrome extension:
Export the Model (Python)
from ultralytics import YOLO # Load the YOLOv8 model model = YOLO("yolov8n.pt") # Export the model to TF.js format model.export(format="tfjs") # creates '/yolov8n_web_model'Load and Use the Model (JavaScript)
import * as tf from '@tensorflow/tfjs'; async function loadModel() { const model = await tf.loadGraphModel('/path/to/yolov8n_web_model/model.json'); return model; } async function runInference(imageElement) { const model = await loadModel(); const tensor = tf.browser.fromPixels(imageElement).expandDims(0); const predictions = await model.executeAsync(tensor); console.log(predictions); } // Example usage const imageElement = document.getElementById('input-image'); runInference(imageElement);Additional Resources
For more detailed instructions on deploying TF.js models, you can refer to:
If you encounter any issues, please ensure you are using the latest version of the Ultralytics package and provide a minimum reproducible example if possible. This helps us diagnose and resolve any problems more effectively. You can find more information on creating a reproducible example here.
Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊
-
wuhonglei
i have the same problem about how to get the
boxes,scoresandclassesfrom thepredictionsvariable, predictions is an object ,not an array, there is no property named boxes, scores and classes, does anyone know how to solve this, very appreciate 😄my html code is below:
<!DOCTYPE html> <html> <head> <title>YOLOv8 Model Inference</title> </head> <body> <!-- 用于显示输入图片和绘制推理结果 --> <canvas id="input-image" width="640" height="640"></canvas> <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script> <script> async function runModel() { const model = await tf.loadGraphModel("./best_web_model/model.json"); // 1. 准备图片数据(这里使用示例图片) const imgElement = document.getElementById("input-image"); const tensor = tf.browser .fromPixels(imgElement) .expandDims(0) .toFloat() .div(tf.scalar(255)); // 2. 进行推理 const predictions = await model.executeAsync([tensor]); } </script> <script> // 获取 canvas 元素 const canvas = document.getElementById("input-image"); const ctx = canvas.getContext("2d"); // 创建新的图片对象 const img = new Image(); // 图片加载完成后绘制到 canvas 上 img.onload = function () { ctx.drawImage(img, 0, 0, canvas.width, canvas.height); runModel(); }; // 设置图片的来源 img.src = "./imgs/20217191442296037.jpg"; // 替换为您的图片路径 </script> </body> </html>the output of predictions is below:
{ "kept": false, "isDisposedInternal": false, "shape": [ 1, 5, 8400 ], "dtype": "float32", "size": 42000, "strides": [ 42000, 8400 ], "dataId": { "id": 1212 }, "id": 701, "rankType": "3", "scopeId": 879 }
Glenn JocherUltralytics(已编辑)Update: the TF.js export is deprecated. As of Ultralytics 8.4.83,
format="tfjs"exports a LiteRT.tflitemodel instead of a*_web_model/model.json; useformat="litert"and see the LiteRT guide. The TF.js snippets below are legacy.Hi @wuhonglei,
Thank you for your detailed question and for sharing your code! 😊
To extract the
boxes,scores, andclassesfrom thepredictionsobject in TensorFlow.js, you need to process the tensor outputs correctly. Thepredictionsobject you have is a tensor, and you can use TensorFlow.js methods to extract the necessary information.Here’s how you can modify your code to get the
boxes,scores, andclasses:Updated JavaScript Code
import * as tf from '@tensorflow/tfjs'; async function runModel() { const model = await tf.loadGraphModel('./best_web_model/model.json'); // Prepare image data const imgElement = document.getElementById('input-image'); const tensor = tf.browser.fromPixels(imgElement).expandDims(0).toFloat().div(tf.scalar(255)); // Run inference const predictions = await model.executeAsync(tensor); // Extract boxes, scores, and classes const [boxes, scores, classes] = await Promise.all([ predictions[0].array(), predictions[1].array(), predictions[2].array() ]); console.log('Boxes:', boxes); console.log('Scores:', scores); console.log('Classes:', classes); // Optionally, draw boxes on the canvas drawBoxes(imgElement, boxes, scores, classes); } function drawBoxes(imgElement, boxes, scores, classes) { const canvas = document.getElementById('input-image'); const ctx = canvas.getContext('2d'); ctx.clearRect(0, 0, canvas.width, canvas.height); ctx.drawImage(imgElement, 0, 0, canvas.width, canvas.height); boxes.forEach((box, i) => { const [y, x, height, width] = box; ctx.strokeStyle = 'red'; ctx.lineWidth = 2; ctx.strokeRect(x, y, width, height); ctx.fillStyle = 'red'; ctx.fillText(`Class: ${classes[i]}, Score: ${scores[i].toFixed(2)}`, x, y > 10 ? y - 5 : 10); }); } // Load image and run model const img = new Image(); img.onload = function () { const canvas = document.getElementById('input-image'); const ctx = canvas.getContext('2d'); ctx.drawImage(img, 0, 0, canvas.width, canvas.height); runModel(); }; img.src = './imgs/20217191442296037.jpg'; // Replace with your image pathExplanation
- Model Loading: Load the model using
tf.loadGraphModel. - Image Preparation: Convert the image to a tensor and normalize it.
- Run Inference: Execute the model and get the predictions.
- Extract Data: Use
array()to convert tensors to JavaScript arrays. - Draw Boxes: Optionally, draw the bounding boxes on the canvas.
This should help you extract and visualize the
boxes,scores, andclassesfrom your predictions. If you encounter any issues, please ensure you are using the latest version of TensorFlow.js and the Ultralytics package.For more detailed information, you can refer to our TF.js integration guide.
Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊
- Model Loading: Load the model using
wuhonglei
thanks very much for your detailed response, In order to better assist future developers, I will paste the code that I have successfully run in the end。 My business background: i use the yolov8 to detect the penguin in the image(object detection ). my final html code:
<!DOCTYPE html> <html> <head> <title>YOLOv8 Object Detection with TensorFlow.js</title> <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script> </head> <body> <input type="file" id="upload" accept="image/*" /> <canvas id="canvas"></canvas> <script> async function loadModel() { // 加载模型 const model = await tf.loadGraphModel("./best_web_model/model.json"); return model; } async function predict(model, imageElement) { // 将图像转换为张量 const tensor = tf.browser .fromPixels(imageElement) .resizeNearestNeighbor([640, 640]) // 调整大小到模型输入大小 .toFloat() .expandDims(0) .div(tf.scalar(255.0)); // inference const predictions = await model.executeAsync(tensor); // predictions shape is [1, 5, 8400] const reshapedPredictions = predictions.reshape([5, 8400]); const boxesTensor = reshapedPredictions.slice([0, 0], [4, 8400]); const scoresTensor = reshapedPredictions .slice([4, 0], [1, 8400]) .squeeze(); // Assuming that the category label of each box is fixed (you can adjust it as needed) const classesTensor = tf.fill([8400], 0); return { boxes: boxesTensor, scores: scoresTensor, classes: classesTensor, }; } function drawBoxes(imageElement, canvas, boxes, scores, classes) { const ctx = canvas.getContext("2d"); canvas.width = imageElement.width; canvas.height = imageElement.height; ctx.drawImage(imageElement, 0, 0, canvas.width, canvas.height); const labels = ["企鹅"]; // Replace with actual label const validBoxes = []; for (let i = 0; i < scores.length; i++) { if (scores[i] > 0.5) { const x_c = boxes[0][i] / 640; const y_c = boxes[1][i] / 640; const w = boxes[2][i] / 640; const h = boxes[3][i] / 640; const x = x_c - w / 2; const y = y_c - h / 2; validBoxes.push([x, y, w, h, scores[i], classes[i]]); } } validBoxes.forEach((box) => { const [x, y, w, h, score, className] = box; ctx.strokeStyle = "red"; ctx.lineWidth = 2; ctx.strokeRect( x * canvas.width, y * canvas.height, w * canvas.width, h * canvas.height ); const text = `${className}: ${(score * 100).toFixed(2)}%`; ctx.fillText(text, x * canvas.width, y * canvas.height - 10); }); } document .getElementById("upload") .addEventListener("change", async (event) => { const file = event.target.files[0]; const imageElement = new Image(); const reader = new FileReader(); reader.onload = () => { imageElement.src = reader.result; imageElement.onload = async () => { const canvas = document.getElementById("canvas"); const model = await loadModel(); const { boxes, scores, classes } = await predict( model, imageElement ); drawBoxes( imageElement, canvas, boxes.arraySync(), scores.arraySync(), classes.arraySync() ); }; }; reader.readAsDataURL(file); }); </script> </body> </html>the
predictionsvariable is a tensor, the format is as below:
the chrome html running result is below:

Glenn JocherUltralytics(已编辑)Update: the TF.js export is deprecated. As of Ultralytics 8.4.83,
format="tfjs"exports a LiteRT.tflitemodel instead of a*_web_model/model.json; useformat="litert"and see the LiteRT guide. The TF.js snippets below are legacy.Hi @wuhonglei,
Thank you for sharing your detailed implementation and the final working code! It's fantastic to see how you've successfully integrated YOLOv8 with TensorFlow.js for penguin detection in your web application. 🐧
For those looking to extract
boxes,scores, andclassesfrom thepredictionstensor, your approach is spot on. Here's a concise summary and a few additional tips for clarity:Key Steps:
-
Load the Model:
const model = await tf.loadGraphModel("./best_web_model/model.json"); -
Prepare the Image:
const tensor = tf.browser.fromPixels(imageElement) .resizeNearestNeighbor([640, 640]) .toFloat() .expandDims(0) .div(tf.scalar(255.0)); -
Run Inference:
const predictions = await model.executeAsync(tensor); -
Extract Predictions:
const reshapedPredictions = predictions.reshape([5, 8400]); const boxesTensor = reshapedPredictions.slice([0, 0], [4, 8400]); const scoresTensor = reshapedPredictions.slice([4, 0], [1, 8400]).squeeze(); const classesTensor = tf.fill([8400], 0); // Adjust as needed -
Draw Boxes:
function drawBoxes(imageElement, canvas, boxes, scores, classes) { const ctx = canvas.getContext("2d"); canvas.width = imageElement.width; canvas.height = imageElement.height; ctx.drawImage(imageElement, 0, 0, canvas.width, canvas.height); const labels = ["企鹅"]; // Replace with actual label const validBoxes = []; for (let i = 0; i < scores.length; i++) { if (scores[i] > 0.5) { const x_c = boxes[0][i] / 640; const y_c = boxes[1][i] / 640; const w = boxes[2][i] / 640; const h = boxes[3][i] / 640; const x = x_c - w / 2; const y = y_c - h / 2; validBoxes.push([x, y, w, h, scores[i], classes[i]]); } } validBoxes.forEach((box) => { const [x, y, w, h, score, className] = box; ctx.strokeStyle = "red"; ctx.lineWidth = 2; ctx.strokeRect( x * canvas.width, y * canvas.height, w * canvas.width, h * canvas.height ); const text = `${className}: ${(score * 100).toFixed(2)}%`; ctx.fillText(text, x * canvas.width, y * canvas.height - 10); }); }
Additional Tips:
- Model Optimization: Ensure your model is optimized for TF.js to improve performance.
- Threshold Adjustment: Adjust the score threshold (
0.5in this case) based on your application's requirements. - Class Labels: Replace the placeholder labels with actual class names if you have multiple classes.
For more detailed information on deploying YOLOv8 models with TensorFlow.js, you can refer to our TF.js integration guide.
If you encounter any issues or have further questions, feel free to reach out. We're here to help! 😊
3-