开始使用

使用 Ultralytics YOLO26 进行实例分割和跟踪

什么是实例分割?#

实例分割是一种计算机视觉任务,用于在像素级别识别并勾勒图像中的单个对象。与仅按类别对像素进行分类的语义分割不同,实例分割会为每个对象实例赋予唯一标签并精确勾勒其边界,因此对于医学成像、自动驾驶和工业自动化等需要深入空间理解的应用至关重要。

Ultralytics YOLO26 能够精确勾勒对象边界,同时保持 YOLO 模型一贯的速度和效率。

Ultralytics 软件包提供两种实例分割跟踪类型:

  • 使用类别对象进行实例分割:每个类别对象都会被分配一种独特的颜色,以便清晰区分。

  • 使用对象轨迹进行实例分割:每条轨迹都以不同的颜色表示,方便在视频帧中识别和跟踪对象。



观看: 使用 Ultralytics YOLO26 进行实例分割和目标跟踪

样本#

实例分割实例分割 + 对象跟踪
Ultralytics 实例分割Ultralytics 实例分割与对象跟踪
Ultralytics 实例分割Ultralytics 实例分割与目标跟踪

使用 YOLO26 进行分割和跟踪#

InstanceSegmentation 解决方案会在每一帧上运行 YOLO26 分割模型,绘制每个对象的掩码,并为每个实例单独着色,让你能够在视频中跟踪各个对象。加载 -seg 模型(例如 YOLO26n-seg),然后通过 Python API 或 CLI 在输入源上运行模型。

使用 Ultralytics YOLO 进行实例分割
# Instance segmentation using Ultralytics YOLO26
yolo solutions isegment show=True

# Pass a source video
yolo solutions isegment source="path/to/video.mp4"

# Monitor the specific classes
yolo solutions isegment classes="[0, 5]"

InstanceSegmentation 参数#

下面是 InstanceSegmentation 参数表:

参数类型默认值说明
modelstrNoneUltralytics YOLO 模型文件的路径。
line_widthint2解决方案绘制的框、关键点和计数的线条粗细。
verboseboolTrue启用逐帧日志,记录输入形状、类别计数和处理速度。跟踪调用本身始终不会输出日志。

你也可以在 InstanceSegmentation 解决方案中使用 track 参数:

参数类型默认值说明
trackerstr'tracktrack.yaml'指定要使用的跟踪算法。内置选项:botsort.yaml、bytetrack.yaml、ocsort.yaml、deepocsort.yaml、fasttrack.yaml、tracktrack.yaml。
conffloat0.25设置检测的置信度阈值;较低的值允许跟踪更多物体,但可能会包含误检。
ioufloat0.5设置用于筛除重叠检测结果的交并比 (IoU) 阈值。
classeslistNone按类别索引筛选结果。例如,classes=[0, 2, 3] 只跟踪指定类别。
devicestrNone指定推理设备(例如 cpu、cuda:0 或 0)。用户可以选择 CPU、特定 GPU 或其他计算设备来运行模型。
imgszint or tuple640目标推理图像尺寸:正方形图像使用整数,或使用 (height, width) 对。图像尺寸会调整为模型步幅的整数倍。
max_detint300跟踪前每帧保留的最大检测数量,用于限制解决方案在密集场景中处理的物体数量。
quantizeint or strNone推理精度:对于非 CPU 上运行的 PyTorch 和 TorchScript 模型,使用 16 (FP16) 或 32/unset (FP32) 进行选择;其他格式使用其模型文件和运行时所选的精度。此参数替代已弃用的 half 参数。

此外,还可以使用以下可视化参数:

参数类型默认值说明
showboolFalse如果 True,则在窗口中显示带标注的图像或视频。适用于在开发或测试期间即时查看结果。
show_confboolTrue在每个检测结果的标签旁显示置信度分数,帮助了解模型对该检测结果的确信程度。
show_labelsboolTrue在可视化输出中显示每个检测结果的标签,便于立即了解检测到的对象。
show_boxesboolTrue在检测到的对象周围绘制边界框,便于在图像或视频帧中直观识别和定位对象。

实例分割的应用#

YOLO26 实例分割在各行各业都有许多实际应用:

废弃物管理与回收#

YOLO26 可用于废弃物管理设施,识别并分类不同类型的材料。该模型能够高精度地分割塑料废弃物、纸板、金属和其他可回收物,使自动化分拣系统更高效地处理废弃物。这一点尤其重要,因为全球每年产生的 70 亿吨塑料废弃物中,只有约 10% 得到回收。

自动驾驶汽车#

在自动驾驶汽车中,实例分割有助于在像素级识别和跟踪行人、车辆、交通标志及其他道路元素。要做出导航和安全决策,准确理解环境至关重要。YOLO26 的实时性能使其非常适合这类对时效性要求很高的应用。

医学成像#

实例分割可在医学扫描图像中识别并勾勒肿瘤、器官或细胞结构。YOLO26 能够精确勾勒对象边界,因此可用于医学诊断和治疗规划。

建筑工地监控#

在施工现场,实例分割可以跟踪重型机械、工人和材料。通过监控设备位置并检测工人何时进入危险区域,它有助于保障安全,同时还能优化工作流程和资源分配。

常见问题#

  • 要使用 Ultralytics YOLO26 执行实例分割,请使用 YOLO26 分割模型初始化 InstanceSegmentation 解决方案,然后通过该方案处理视频帧。下面是一个简化的代码示例:

    import cv2
    
    from ultralytics import solutions
    
    cap = cv2.VideoCapture("path/to/video.mp4")
    assert cap.isOpened(), "Error reading video file"
    
    # 视频写入器
    w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))
    video_writer = cv2.VideoWriter("instance-segmentation.avi", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))
    
    # 初始化实例分割
    isegment = solutions.InstanceSegmentation(
        show=True,  # 显示输出
        model="yolo26n-seg.pt",  # model="yolo26n-seg.pt" 用于使用 YOLO26 进行对象分割。
    )
    
    # 处理视频
    while cap.isOpened():
        success, im0 = cap.read()
        if not success:
            print("Video frame is empty or processing is complete.")
            break
        results = isegment(im0)
        video_writer.write(results.plot_im)
    
    cap.release()
    video_writer.release()
    cv2.destroyAllWindows()
  • 实例分割会识别并勾勒图像中的单个对象,为每个对象分配唯一的标签和掩码。目标跟踪则在此基础上,为视频帧中的对象分配一致的 ID,从而持续跟踪同一对象。将两者结合起来(如 YOLO26 的实现所示),你就可以在视频中分析对象的运动和行为,同时保留精确的边界信息。

  • 与 Mask R-CNN 或 Faster R-CNN 等两阶段模型相比,Ultralytics YOLO26 具有实时性能、出色的准确率和易用性。YOLO26 只需单次处理图像(单阶段检测),速度快得多,同时保持较高的精度。它还可与 Ultralytics Platform 集成,用于管理模型、数据集和训练流程。对于需要兼顾速度和准确率的应用,YOLO26 实现了出色的平衡。

  • 是的,Ultralytics 提供了多个适用于训练 YOLO26 实例分割模型的数据集,包括 COCO-Seg、COCO8-Seg(较小的子集,适合快速测试)、Package-Seg 和 Crack-Seg。这些数据集提供实例分割任务所需的像素级标注。对于更专业的应用,你也可以按照 Ultralytics 格式创建自定义数据集。完整的数据集信息和使用说明请参阅 Ultralytics 数据集文档。

评论99

提出问题、分享结果,或帮助其他人。

  1. CatFatOwGitHub

    Hello, I am trying to analyze a video and see if features grow/shrink over time. I am currently trying to use instance segmentation with object tracking; however, specifying save_txt=True parameter in model.track saves every single prediction into a single huge txt file. Trying to count the frames by going through this large txt apparently doesn't work as the class ID are not in order and sorting it would lose all information.

    Is there any way for Ultralytics to save the txt files by frame number? For example, frame1.txt only contains the predictions of the first frame.

    Any help would be greatly appreciated!

    1
    1. Glenn JocherUltralytics

      For per-frame TXT saving with tracking, you'll need to process results in your video loop and write custom frame-specific files. Here's a minimal adaptation of the Instance Segmentation and Tracking example:

      frame_count = 0
      while cap.isOpened():
          ret, im0 = cap.read()
          if not ret: break
          
          results = model.track(im0, persist=True)
          
          # Save current frame's tracked objects
          with open(f'frame{frame_count}.txt', 'w') as f:
              for box, mask, track_id in zip(results[0].boxes, results[0].masks, results[0].boxes.id.int().cpu().tolist()):
                  f.write(f"{track_id} {box.cls.item()} {box.xywhn.tolist()} {mask.xy[0].tolist()}\n")
          
          frame_count += 1

      This creates individual frameX.txt files with tracking IDs, class IDs, normalized boxes, and mask coordinates. The official documentation shows similar frame processing in the Instance Segmentation with Object Tracking examples.

    2. Glenn JocherUltralytics

      For per-frame TXT saving during tracking, you'll need to process each frame individually in your video loop as shown in the Instance Segmentation with Object Tracking example. Here's a minimal adaptation that creates frame-specific files:

      frame_count = 0
      while cap.isOpened():
          ret, im0 = cap.read()
          if not ret: break
          
          results = model.track(im0, persist=True)
          
          with open(f'frame{frame_count}.txt', 'w') as f:
              for box, mask, track_id in zip(results[0].boxes, results[0].masks, results[0].boxes.id.int().cpu().tolist()):
                  f.write(f"{track_id} {box.cls.item()} {box.xywhn.tolist()} {mask.xy[0].tolist()}\n")
          
          frame_count += 1

      This creates individual frameX.txt files with tracking IDs, class IDs, normalized boxes, and mask coordinates per frame. The approach mirrors the frame processing shown in our official tracking documentation.

  2. pedmindsetGitHub

    Been trying to figure this out for a while now. I want to ask, can I use object counting in regions with instance sementation and tracking? You want to count vehicles that cross two specific regions, like so Object 001 crossed region A and region B. Is this possible to do

    1
    1. Glenn JocherUltralytics

      @pedmindset yes, it is possible to combine object counting in regions with instance segmentation and tracking using Ultralytics YOLO11. You can use the track method to assign consistent IDs to objects across frames and define regions to monitor their crossings. For guidance, refer to the segmentation and tracking examples in the Instance Segmentation and Tracking Guide. Let us know if you need further clarification!

  3. DP1701GitHub

    Cool, but I can't define an imgsz via solutions.InstanceSegmentation(…). Is there another option for this?

    1. Glenn JocherUltralytics

      solutions.InstanceSegmentation is a higher-level wrapper and doesn’t currently expose imgsz directly; if you need to control input size, run segmentation+tracking via the core YOLO(...).track() API (supports imgsz) as shown in the Tracking mode docs and keep using the Instance Segmentation and Tracking guide for solution-level usage.

      from ultralytics import YOLO
      
      model = YOLO("yolo26n-seg.pt")
      results = model.track(source="path/to/video.mp4", imgsz=640, show=True, tracker="botsort.yaml")
  4. CatFatOwGitHub

    Hello, I noticed that when using tracking mode compared to normal segmentation prediction, the number of features predicted decreased by quite a lot for feature dense videos even if I use the same parameters of confidence and maximum detection?

    1. Glenn JocherUltralytics

      @CatFatOw great observation! This is actually expected behavior when using tracking mode compared to plain segmentation. Tracking algorithms apply additional filtering beyond confidence thresholds to maintain stable tracks, which can reduce the total number of detections.

      Here are a few things that might help increase detections while maintaining tracking:

      # Try adjusting these tracking-specific parameters
      results = model.track(
          frame, 
          persist=True,
          conf=0.3,  # Lower confidence threshold
          iou=0.5,   # Adjust IoU threshold for NMS
          max_det=1000,  # Increase max detections
          tracker="bytetrack.yaml"  # Try different trackers
      )

      You can also experiment with different tracking algorithms - ByteTrack tends to be more permissive with new track creation compared to BoTSORT. The trade-off is between detection sensitivity and tracking stability, where tracking prioritizes consistent object IDs over detecting every possible instance.

      If you need maximum detection sensitivity, consider running both modes and using tracking results only when you need the temporal consistency.

  5. Mauricio566GitHub

    Hello. Please, I need your help. I know exactly what segmenting an object is and how to do it. But I don't know what to do with an already segmented object or pixel mask. You can find a lot of information online about "how to segment a model or fit it to your dataset," but they don't tell you what to do with those masks in real projects. Can you give me some ideas or help, please? for r in results:
    if r.masks is not None: mask = r.masks.data[0].cpu().numpy() there is the mask but honestly I don't know what to do. thanks

  6. e-roeGitHub

    Hello Glenn, thanks for the example! Actually, I had already figured it out—it was more of a suggestion for the documentation examples. []´s

    1. Glenn JocherUltralytics

      Thank you for the suggestion! We always welcome community contributions to improve our documentation through GitHub Pull Requests. Feel free to submit any specific improvements you'd like to see in the Instance Segmentation with Object Tracking guide.

      1
  7. e-roeGitHub

    Hello, Wouldn't it be good to have a complete example showing how to extract the track ID, contour points, class, and confidence? These methods that handle everything automatically end up being of little practical use, and finding this information in the documentation is quite challenging—it's often easier to find from third-party sources. []´s

    1. Glenn JocherUltralytics(已编辑)

      @e-roe thanks for your feedback! You can access track IDs, classes, masks, and confidence scores directly from the results object. Here's a concise example showing how to extract these details:

      results = isegment(im0)
      for i in range(len(isegment.clss)):
          track_id = isegment.track_ids[i]
          class_name = isegment.names[isegment.clss[i]]
          confidence = isegment.confs[i]
          mask = isegment.masks[i].data[0].numpy()  # Binary mask array
          contours, _ = cv2.findContours(mask.astype(np.uint8), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

      For full implementation details, see the InstanceSegmentation reference in our documentation. Let us know if you need further clarification!

  8. expen6veGitHub

    Hi! I tried training a pretrained yolov8m-seg model on our custom 1 class dataset consisting of 1,307 images with 5 augmentations like:

    Flip: Horizontal, Vertical Rotation: Between -15° and +15° Saturation: Between -25% and +25% Blur: Up to 1.4px Noise: Up to 1.49% of pixels

    with these training configurations:

    # Train the model
    results = model.train(
        data="data6.yaml",
        epochs=250,
        imgsz=320,
        batch=8,
        workers=4,
        device=device,
        patience=18,
        copy_paste=0.1  # Add this line to enable copy-paste augmentation
    )

    this is my 16th model training as i tried testing out from 200+ images that and so on. In my val plots, i always get bad results. What can i further do to make a good model? it stopped training at 128 with the results below:

    epoch,time,train/box_loss,train/seg_loss,train/cls_loss,train/dfl_loss,metrics/precision(B),metrics/recall(B),metrics/mAP50(B),metrics/mAP50-95(B),metrics/precision(M),metrics/recall(M),metrics/mAP50(M),metrics/mAP50-95(M),val/box_loss,val/seg_loss,val/cls_loss,val/dfl_loss,lr/pg0,lr/pg1,lr/pg2

    126,4123.96,0.47628,0.76702,0.30532,0.85667,0.9862,0.96565,0.98972,0.94445,0.9775,0.97566,0.98923,0.83189,0.35178,0.67432,0.26286,0.89191,0.00101,0.00101,0.00101 127,4156.45,0.47613,0.76458,0.30439,0.85872,0.96877,0.98339,0.98913,0.94227,0.98701,0.96962,0.99053,0.83626,0.34228,0.67744,0.25083,0.88958,0.00100208,0.00100208,0.00100208

    1. Glenn JocherUltralytics(已编辑)

      @expen6ve thank you for sharing your training details. For improved segmentation results with YOLO26, consider these adjustments:

      1. Dataset Size: 1.3k images may be insufficient for robust segmentation. Aim for ≥5k high-quality annotated images.
      2. Augmentation Tuning: Reduce rotation range (try ±5°) and remove noise augmentation, which can degrade mask precision.
      3. Hyperparameters: Increase imgsz to 640 (standard for segmentation tasks) and lower learning rate to 0.0001 with cosine scheduler.
      4. Model Selection: Try YOLO26s-seg instead of medium-sized models for better generalization on limited data.

      For detailed guidance, see the Segmentation Training Tips in our documentation.

  9. ShreyasChhetriGitHub

    Hi can we implement a Reinforcement Learning function up above the Yolo training function for more accuracy can it happen ?

    1. Glenn JocherUltralytics

      @ShreyasChhetri integrating reinforcement learning (RL) with YOLO training is an interesting idea, but YOLO models are not inherently designed for RL workflows. While it is theoretically possible, it would require substantial custom development outside the scope of the provided Ultralytics framework. You might consider exploring hybrid approaches combining RL and computer vision if your use case demands it. For standard training, refer to the existing YOLO training documentation: https://docs.ultralytics.com/modes/train.

  10. ZhangAiyiGitHub

    I have a question when run the code for Instance Segmentation with Object Tracking. The pycharm said UnicodeEncodeError: 'gbk' codec can't encode character '\u274c' in position 401: illegal multibyte sequence. Does anyone have the similar question? Thank you.

    1. Glenn JocherUltralytics

      @ZhangAiyi it seems like you're encountering a Unicode encoding issue. You can try setting your environment's encoding to UTF-8 in PyCharm to resolve this. If the problem persists, please check the PyCharm settings or your system's locale configuration.