开始使用

将 YOLO26 模型导出为 TorchScript,以便快速部署

PyTorch 正在逐步淘汰 TorchScript

PyTorch 已将 TorchScript 标记为弃用,并正逐步移除其功能。对于新的移动端和边缘端部署,请使用受支持的 ExecuTorch 集成。Ultralytics 仍保留常规 TorchScript 导出,以兼容旧版 C++。

在不使用 Python 的 C++ 环境中部署计算机视觉模型,需要一种可移植的序列化表示。TorchScript 为旧版 LibTorch 应用提供了这种兼容性。

导出为 TorchScript 格式,可序列化你的 Ultralytics YOLO26 模型,实现跨平台兼容并简化部署。本指南将介绍如何将 YOLO26 模型导出为 TorchScript 格式,以便在更多类型的应用中使用。

为什么应该导出为 TorchScript?#

TorchScript 模型序列化和部署工作流程概览

TorchScript 由 PyTorch 的创建者开发,是一款用于在各种平台上优化和部署 PyTorch 模型的工具。将 YOLO26 模型导出为 TorchScript,有助于将模型从研究转化为实际应用。TorchScript 是 PyTorch 框架的一部分,可让 PyTorch 模型在不支持 Python 的环境中使用,从而简化这一转变。

这一过程包含两种技术:跟踪和脚本化。跟踪会记录模型执行期间的操作,而脚本化则允许使用 Python 的一个子集来定义模型。这些技术让 YOLO26 等模型能够脱离其常用的 Python 环境运行。

TorchScript 脚本化与跟踪对比

TorchScript 模型还可以通过算子融合和内存使用优化等技术进行优化,从而确保高效执行。导出为 TorchScript 的另一项优势是,它有可能加速模型在各种硬件平台上的执行。它会生成独立且可用于生产环境的 PyTorch 模型表示,可集成到 C++ 环境中。

TorchScript 模型的主要特性#

TorchScript 是 PyTorch 生态系统的重要组成部分,提供了优化和部署深度学习模型的功能。

TorchScript 主要特性概览

以下是使 TorchScript 成为开发者实用工具的主要特性:

  • 静态图执行:TorchScript 使用模型计算的静态图表示,这与 PyTorch 的动态图执行不同。在静态图执行中,计算图会在实际执行前定义并编译一次,从而提升推理期间的性能。

  • 模型序列化:TorchScript 支持将 PyTorch 模型序列化为与平台无关的格式。无需原始 Python 代码即可加载序列化模型,从而支持在不同的运行时环境中部署。

  • JIT 编译:TorchScript 使用即时 (JIT) 编译,将 PyTorch 模型转换为经过优化的中间表示。JIT 会编译模型的计算图,从而让模型能够在目标设备上高效运行。

  • 渐进式转换:TorchScript 提供渐进式转换方式,让你可以逐步将 PyTorch 模型的各个部分转换为 TorchScript。当处理复杂模型或希望优化代码的特定部分时,这种灵活性尤其有用。

TorchScript 的部署选项#

在查看将 YOLO26 模型导出为 TorchScript 格式的代码之前,先来了解一下 TorchScript 模型的典型用途。

TorchScript 为机器学习模型提供多种部署选项,例如:

  • C++ API:TorchScript 最常见的用例是使用 LibTorch C++ API,在 C++ 应用中直接加载并运行经过优化的 TorchScript 模型。这非常适合 Python 不适用或不可用的生产环境。C++ API 能够以较低开销高效运行 TorchScript 模型,充分发挥性能潜力。

  • 移动端部署:如需在移动设备上实现低延迟、离线推理和数据隐私,请使用 ExecuTorch,它是 PyTorch 对 TorchScript Mobile 的替代方案。

  • 云端部署:TorchScript 模型可以使用 NVIDIA Triton Inference Server 等解决方案部署到云服务器。它提供模型版本管理、批处理和指标监控等功能,支持在生产环境中进行可扩展部署。使用 TorchScript 进行云端部署,可以让你通过 API 或其他 Web 服务访问模型。

支持的任务#

TorchScript 导出支持 Ultralytics 的全部七项任务。语义分割和深度估计仅适用于 YOLO26,因为只有 YOLO26 系列包含这两种检测头。

任务YOLOv8YOLO11YOLO26
检测✅✅✅
分割✅✅✅
语义分割❌❌✅
深度估计❌❌✅
分类✅✅✅
姿态✅✅✅
OBB✅✅✅

导出为 TorchScript:转换 YOLO26 模型#

将 YOLO26 模型导出为 TorchScript,可让模型更轻松地在 Python 环境之外运行,例如在基于 LibTorch 构建的 C++ 应用中运行。

安装#

要安装所需软件包,请运行:

安装
# Install the required package for YOLO26
pip install ultralytics

有关安装流程的详细说明和最佳实践,请查看我们的 Ultralytics 安装指南。安装 YOLO26 所需的软件包时,如果遇到困难,请查阅我们的常见问题指南,获取解决方案和建议。

用法#

所有 Ultralytics YOLO26 模型均支持开箱即用的导出,便于你将模型集成到首选的部署工作流中。你可以查看支持的导出格式和配置选项的完整列表,为你的应用选择最佳配置。

TorchScript 格式支持导出、预测和验证模式。导出模型后,加载导出的模型即可运行推理或验证其准确率。

导出
from ultralytics import YOLO

# 加载 YOLO26 模型
model = YOLO("yolo26n.pt")

# 将模型导出为 TorchScript 格式
model.export(format="torchscript")  # 创建 'yolo26n.torchscript'
预测
from ultralytics import YOLO

# 加载导出的 TorchScript 模型
model = YOLO("yolo26n.torchscript")

# 运行推理
results = model("https://ultralytics.com/images/bus.jpg")
验证
from ultralytics import YOLO

# 加载导出的 TorchScript 模型
model = YOLO("yolo26n.torchscript")

# 在 COCO8 数据集上验证准确率
metrics = model.val(data="coco8.yaml")

导出参数#

参数类型默认值说明
formatstr'torchscript'导出模型的目标格式,用于定义与各种部署环境的兼容性。
imgszint 或 tuple640模型输入所需的图像尺寸。正方形图像可使用整数;若要指定具体尺寸,则可使用元组 (height, width)。
dynamicboolFalse允许使用动态输入尺寸,从而更灵活地处理不同图像尺寸。
quantizeint 或 strNone量化精度:16 (FP16) 需要使用 device=0 进行 GPU 导出;32/未设置时使用 FP32。此设置替代了已弃用的 half 标志。
nmsbool,可选None选择原始输出(None,默认)、嵌入式 NMS(True)或无 NMS 检测头(False)。
batchint1指定导出模型的批量推理大小,或导出模型在 predict 模式下可同时处理的最大图像数。
devicestrNone指定导出设备:GPU(device=0)、CPU(device=cpu)、Apple 芯片的 MPS(device=mps)。

有关导出流程的更多详情,请访问 Ultralytics 导出文档页面。

部署导出的 YOLO26 TorchScript 模型#

成功将 Ultralytics YOLO26 模型导出为 TorchScript 格式后,你就可以部署这些模型了。运行 TorchScript 模型的首选且推荐的第一步,是使用 YOLO("model.torchscript") 方法,如前面的用法代码片段所示。若要详细了解如何在其他环境中部署 TorchScript 模型,请参阅以下资源:

  • 了解移动端部署:使用 ExecuTorch 独立的 torch.export() → .pte 流程进行当前 PyTorch 移动端部署。

  • 掌握服务器端部署:了解如何使用 NVIDIA Triton Inference Server 的 PyTorch 后端提供 TorchScript 模型服务,实现可扩展、高效的模型服务。

  • 实现 C++ 部署:深入了解《在 C++ 中加载 TorchScript 模型教程》,将 TorchScript 模型集成到 C++ 应用中,以提升性能和灵活性。

总结#

本指南介绍了如何将 Ultralytics YOLO26 模型导出为 TorchScript 格式。按照所提供的说明操作,你就能优化 YOLO26 模型的性能,并灵活地将模型部署到各种平台和环境中。

如需了解更多用法详情,请访问 TorchScript 官方文档。

常见问题#

  • 将 Ultralytics YOLO26 模型导出为 TorchScript,可以灵活地进行跨平台部署。TorchScript 是 PyTorch 生态系统的一部分,可对模型进行序列化,使其能够在不支持 Python 的环境中运行。因此,它适用于在 C++ 环境中部署模型。

  • 要将 YOLO26 模型导出为 TorchScript,可以使用以下示例代码:

    用法
    from ultralytics import YOLO
    
    # 加载 YOLO26 模型
    model = YOLO("yolo26n.pt")
    
    # 将模型导出为 TorchScript 格式
    model.export(format="torchscript")  # 创建 'yolo26n.torchscript'
    
    # 加载导出的 TorchScript 模型
    torchscript_model = YOLO("yolo26n.torchscript")
    
    # 运行推理
    results = torchscript_model("https://ultralytics.com/images/bus.jpg")

    有关导出流程的更多详情,请参阅 Ultralytics 导出文档。

  • 使用 TorchScript 部署 YOLO26 模型有以下几项优势:

    • 可移植性:导出的模型可以在不使用 Python 的 C++ 应用中运行。
    • 优化:TorchScript 支持静态图执行和即时 (JIT) 编译,有助于优化模型性能。
    • 跨语言集成:TorchScript 模型可以集成到其他编程语言中,增强灵活性和可扩展性。
    • 序列化:模型可以序列化,从而支持与平台无关的加载和推理。

    如需了解更多部署信息,请参阅 C++ 部署指南。对于设备端移动部署,PyTorch 现在推荐使用 ExecuTorch,它采用独立的 torch.export() → .pte 流程,而不是 TorchScript。

  • 要安装导出 YOLO26 模型所需的软件包,请使用以下命令:

    安装
    # Install the required package for YOLO26
    pip install ultralytics

    如需详细说明,请访问 Ultralytics 安装指南。如果安装过程中遇到问题,请参阅常见问题指南。

  • 将 YOLO26 模型导出为 TorchScript 格式后,你可以在各种平台上部署它们:

    • C++ API:在需要低开销、高效率的生产环境中使用 LibTorch。
    • 移动端部署:使用 ExecuTorch,这是 PyTorch 支持的替代方案,具有独立的 .pte 导出流程。
    • 云端部署:使用 NVIDIA Triton Inference Server 等服务实现可扩展的服务器端部署。

评论9

提出问题、分享结果,或帮助其他人。

  1. amarnathreddy0201GitHub

    Load the YOLOv8 model

    model = YOLO("yolov8n.pt")

    Export the model to TorchScript format

    model.export(format="torchscript") # creates 'yolov8n.torchscript'

    Load the exported TorchScript model

    torchscript_model = YOLO("yolov8n.torchscript")

    For loading "yolov8n.torchscript" here yolo is used. is it possible to load the model using pytorch and prediction/Segmentation.

    1. Paula DerrengerUltralytics

      @amarnathreddy0201 yes, you can load and use the exported TorchScript model directly with PyTorch. After exporting your YOLOv8 model to TorchScript format, you can load it using torch.jit.load('yolov8n.torchscript') and then perform inference as usual with PyTorch. For detailed instructions, you can refer to the TorchScript documentation.

    2. amarnathreddy0201GitHub(已编辑)

      @pderrenger I written small application for person detection but it's not detecting properly nd bounding boxes is some where else.

      import torch import cv2 import numpy as np

      classes = [ "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush" ]

      model = torch.jit.load("yolov8n.torchscript") image_ori = cv2.imread(r"C:\Users\amarn\OneDrive\Pictures\father mother.jpg") image = cv2.resize(image_ori, (640, 640)) image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) image_tensor = torch.from_numpy(image).float() image_tensor = image_tensor.permute(2, 0, 1) # Change from HWC to CHW format image_tensor = image_tensor / 255.0 # Normalize to [0, 1] image_tensor = image_tensor.unsqueeze(0)

      with torch.no_grad(): results = model(image_tensor)

      results = results[0] # Remove the batch dimension if there's only one image

      boxes = results[:, :4]
      print(len(boxes)) object_confidences = results[:, 4] # Objectness score class_confidences = results[:, 5:] # Class scores

      conf_threshold = 0.5 # Example threshold; adjust as necessary predictions = [] for i in range(len(object_confidences)): if object_confidences[i] > conf_threshold: class_id = torch.argmax(class_confidences[i]).item() class_conf = class_confidences[i][class_id].item()

          if class_conf > conf_threshold:
              
              box_np = boxes[i].cpu().numpy() if boxes[i].is_cuda else boxes[i].numpy()
              x, y, w, h = box_np
              x1 = int((x - w / 2) )
              y1 = int((y - h / 2) )
              x2 = int((x + w / 2) )
              y2 = int((y + h / 2))
              cv2.rectangle(image_ori, (int(x1),int( y1)), (int(x1+ w), int(y1 + h)), (0, 255, 0), 2)

      cv2.imshow('image', image_ori) cv2.waitKey(0)

      classes is from yolo only

      1
    3. Paula DerrengerUltralytics

      It looks like there might be an issue with the bounding box coordinates. Ensure that the coordinates are correctly transformed from the model's output format to the image's coordinate system. Also, verify that the model is correctly loaded and that the input image is preprocessed as expected. If the issue persists, please try updating to the latest version of the Ultralytics package and check if the problem is reproducible. For more details on exporting and using TorchScript models, you can refer to the Ultralytics documentation.

    4. clausMekoGitHub(已编辑)

      @amarnathreddy0201 I could make the boxes work for my One-Item only use case. I guess the tensor structure was not really clear to you as it seemed to me you have profound knowledge. ty for heavy lifting :partying_face:

      @glenn-jocher and @pderrenger if you care what use case I had and could not simply from ultralytics import YOLO: I chain YOLO with other models and have conflicts with imports, especially numpy v1 vs v2. So i figured export all as torchscript and chain them.

      import torch
      import cv2
      
      # Knwoledge: https://github.com/ultralytics/ultralytics/issues/5863
      # Knwoledge: https://github.com/orgs/ultralytics/discussions/9002
      def preprocess_image(image_ori) -> torch.Tensor:
          image = cv2.resize(image_ori, (640, 640))
          image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
          image_tensor = torch.from_numpy(image).float()
          image_tensor = image_tensor.permute(2, 0, 1)  # Change from HWC to CHW format
          image_tensor = image_tensor / 255.0  # Normalize to [0, 1]
          image_tensor = image_tensor.unsqueeze(0)
      
          return image_tensor
      
      def main():
          # Load the TorchScript model
          map_location = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')
          model = torch.jit.load('./best.torchscript', map_location=map_location)
      
          cv2_img = cv2.imread("../obj-det/datasets/test/images/00018_png.rf.d602082fdf686063a7d430aba6dfedde.jpg")
          image = preprocess_image(cv2_img)
      
          with torch.no_grad():
              results: torch.Tensor = model(image.to(map_location))
      
          results = results[0]  # Remove the batch dimension if there's only one image
          boxes = results[:4, :]
          object_confidences = results[4, :]
          conf_threshold = 0.5  # Example threshold; adjust as necessary
      
          dominant_item = torch.argmax(object_confidences)
          if object_confidences[dominant_item] < conf_threshold:
              cv2.imshow('nothing found', cv2_img)
              cv2.waitKey(0)
              return
      
          box_np = boxes[:,dominant_item].cpu().numpy() if boxes[:,dominant_item].is_cuda else boxes[:,dominant_item].numpy()
          x, y, w, h = box_np
          x1 = int((x - w / 2))
          y1 = int((y - h / 2))
          x2 = int(x1 + w)
          y2 = int(y1 + h)
          cv2.rectangle(cv2_img, (x1, y1), (x2, y2), (0, 255, 0), 2)
          cv2.imshow('image', cv2_img)
          cv2.waitKey(0)
      
      if __name__ == '__main__':
          main()
      1
    5. Glenn JocherUltralytics

      Thank you for sharing your use case and detailed implementation! Exporting models as TorchScript for chaining is indeed a smart approach, especially when managing dependency conflicts. If adjustments are needed for bounding boxes or results, ensure the coordinate transformations align with the input image dimensions. For additional guidance on TorchScript usage, feel free to reference Ultralytics TorchScript Docs. Let us know if you encounter further issues!

  2. tryprncpGitHub

    hi, I'm implementing yolov8 model in android gallery's search mechanism. the purpose of yolov8 model is to scan media files and return images that has a bounding box label that matches the search query. i can make it work with yolov5s.torchscript.ptl using org.pytorch:pytorch_android_lite:1.10.0 and org.pytorch:pytorch_android_torchvision_lite:1.10.0 but it wont work with yolov8s.torchscript. i really need help with this.

    1. Glenn JocherUltralytics

      Hey there! 😊 It sounds like you've made great progress integrating YOLOv8 into your Android app's search mechanism! If you've successfully used YOLOv5 models but are facing issues with YOLOv8's TorchScript model, let's try to sort this out.

      First, ensure you've exported the YOLOv8 model to TorchScript correctly using the latest YOLOv8 and PyTorch versions. Here's a quick snippet for exporting:

      from ultralytics import YOLO
      
      # Load YOLOv8 model
      model = YOLO('yolov8s.pt')  # Use the appropriate YOLOv8 variant
      
      # Export to TorchScript
      model.export(format='torchscript')

      This creates a file named yolov8s.torchscript. Ensure that this file is what you're trying to load in your Android app.

      If you're still having trouble after confirming the export process, consider checking the PyTorch Android version compatibility. YOLOv8 may require a newer version of PyTorch Android than YOLOv5, so updating org.pytorch:pytorch_android_lite and org.pytorch:pytorch_android_torchvision_lite to the latest versions might help.

      Lastly, since YOLOv8 is more recent, make sure you're using the latest release of the Ultralytics YOLO library for any compatibility fixes or improvements.

      If these steps don't resolve the issue, feel free to share any error messages or more specifics about the problem you're encountering. I'm here to help!