开始使用

如何使用 Ultralytics YOLO 隔离分割对象

实例分割会为每个检测到的物体生成像素级精准的掩码,因此你可以将每个物体单独从图像中提取出来。本指南将介绍如何使用 Predict Mode 和 OpenCV,将 Ultralytics YOLO 分割结果转换为独立物体,并使用纯黑或透明背景保存为 PNG。



观看: 如何使用 Ultralytics YOLO26 隔离分割对象 | Python 教程 | VisionAI 🚀

为什么要隔离分割对象?#

从图像中提取单个对象,可以支持多种后续工作流:

  • 移除背景,用于产品照片、目录或创意编辑。
  • 按对象裁剪,利用检测结果构建分类数据集。
  • 聚焦处理,让 OCR、颜色分析或测量等后续步骤只处理对象本身,而不处理周围场景。
  • 导出透明 PNG,以便将对象合成到新背景上。

此方法适用于任何 Ultralytics YOLO 分割模型,并包含四个阶段:运行推理 → 提取每个轮廓 → 隔离对象 → 保存结果。

运行分割推理#

安装所需的库,然后加载分割模型,例如 YOLO26n-seg(需要使用 -seg 后缀才能生成掩码),并对源图像运行预测:

from ultralytics import YOLO

# 加载分割模型
model = YOLO("yolo26n-seg.pt")

# 对源图像运行推理
results = model.predict(source="path/to/image.jpg")
没有源图像?YOLO 会使用捆绑的示例图像

如果调用 model.predict() 时未指定 source,Ultralytics 会回退到随软件包附带的示例图像(bus.jpg 和 zidane.jpg),方便你快速测试工作流。

提取对象轮廓#

results 中的每个元素都对应一张图像,遍历结果时每次会得到一个检测对象。对于每个检测结果,复制原始图像,读取类别标签,并将对象的掩码轮廓绘制到空白二值掩码上。该掩码中的白色区域准确标示了属于该对象的像素。

本节和下一节中的代码片段都在下方的检测循环内部运行;完整的可直接复制粘贴的脚本见完整示例。

Binary mask with the segmented object in white on a black background
from pathlib import Path

import cv2
import numpy as np

for r in results:
    img = np.copy(r.orig_img)
    img_name = Path(r.path).stem  # 源图像文件名(不含扩展名)

    # 遍历图像中检测到的每个对象
    for ci, c in enumerate(r):
        label = c.names[c.boxes.cls.tolist().pop()]  # 类别名称

        # 创建二值掩码,并在其上绘制对象轮廓
        b_mask = np.zeros(img.shape[:2], np.uint8)
        contour = c.masks.xy[0].astype(np.int32).reshape(-1, 1, 2)
        cv2.drawContours(b_mask, [contour], -1, (255, 255, 255), cv2.FILLED)
`c.masks.xy[0].astype(np.int32).reshape(-1, 1, 2)` 有什么作用?
  • c.masks.xy[0] 会以 (x, y) 点坐标的形式,返回此单个检测结果中对象的掩码轮廓。
  • .astype(np.int32) 会转换 float32 中的点,因为 OpenCV 的 drawContours() 不接受这种格式。
  • .reshape(-1, 1, 2) 会将这些点重塑为 [N, 1, 2] 布局,这是 drawContours() 所需的格式,其中 N 表示轮廓点的数量。

将 [contour] 与索引 -1 一起传入,会绘制所提供轮廓的所有点,而 cv2.FILLED 会将轮廓内的每个像素填充为白色。

隔离对象#

准备好二值掩码后,将其与原始图像合并。根据你希望使用的背景,常见做法有两种:

选择隔离方式

将掩码转换为三个通道,只保留与对象重叠的像素。轮廓之外的所有区域都会变成黑色:

# 使用黑色背景隔离对象
mask3ch = cv2.cvtColor(b_mask, cv2.COLOR_GRAY2BGR)
isolated = cv2.bitwise_and(mask3ch, img)
Full-size isolated object on a black background
黑色背景上的完整尺寸对象
裁剪到边界框

如果只想保留对象区域,而不是完整尺寸的图像,可以按检测结果的边界框对图像进行切片:

# 边界框坐标
x1, y1, x2, y2 = c.boxes.xyxy.cpu().numpy().squeeze().astype(np.int32)
# 将隔离后的图像裁剪到对象区域
iso_crop = isolated[y1:y2, x1:x2]
Isolated object cropped to its bounding box on a black background
裁剪至边界框的对象
需要保留原始背景的裁剪图吗?

此功能已内置。将 save_crop=True 传递给 predict(),Ultralytics 就会自动保存边界框裁剪图,无需创建掩码。

保存结果(可选)#

如何处理每个隔离出的对象由你决定。一个常见的后续步骤是将对象写入磁盘,以便之后使用:

# Save the isolated object to file
cv2.imwrite(f"{img_name}_{label}-{ci}.png", isolated)

这里,img_name 是源图像文件名(不含扩展名),label 是类别名称,ci 是检测索引,因此同一类别的多个实例会获得唯一的文件名。如果你在上面进行了可选裁剪,请将 isolated 替换为 iso_crop。

完整示例#

下面的脚本将所有步骤整合到一个可运行的代码块中。默认使用黑色背景;将标记的那一行改为 np.dstack([img, b_mask]),即可改为输出透明 PNG:

from pathlib import Path

import cv2
import numpy as np

from ultralytics import YOLO

model = YOLO("yolo26n-seg.pt")
results = model.predict(source="path/to/image.jpg")

for r in results:
    img = np.copy(r.orig_img)
    img_name = Path(r.path).stem

    for ci, c in enumerate(r):
        label = c.names[c.boxes.cls.tolist().pop()]

        # Build a binary mask from the object contour
        b_mask = np.zeros(img.shape[:2], np.uint8)
        contour = c.masks.xy[0].astype(np.int32).reshape(-1, 1, 2)
        cv2.drawContours(b_mask, [contour], -1, (255, 255, 255), cv2.FILLED)

        # Isolate the object (black background)
        mask3ch = cv2.cvtColor(b_mask, cv2.COLOR_GRAY2BGR)
        isolated = cv2.bitwise_and(mask3ch, img)  # transparent PNG: isolated = np.dstack([img, b_mask])

        # Save or add your custom post-processing here
        cv2.imwrite(f"{img_name}_{label}-{ci}.png", isolated)

        # Optional: crop to the bounding box before saving
        # x1, y1, x2, y2 = c.boxes.xyxy.cpu().numpy().squeeze().astype(np.int32)
        # cv2.imwrite(f"{img_name}_{label}-{ci}.png", isolated[y1:y2, x1:x2])

如果要重复使用,可以将循环体封装成函数,以便对多张图像调用。

结论#

现在,你已经掌握了使用 Ultralytics YOLO 隔离分割对象的完整方法:运行推理,根据每个轮廓创建二值掩码,然后在黑色或透明背景上提取对象,并可选择将对象裁剪到其边界框。你可以查看完整的 Segment 任务和 Predict 模式文档,根据自己的类别调整工作流。

常见问题#

  • 加载分割模型,运行推理,根据每个检测结果的轮廓创建二值掩码,然后将掩码与原始图像合并:

    import cv2
    import numpy as np
    
    from ultralytics import YOLO
    
    model = YOLO("yolo26n-seg.pt")
    results = model.predict(source="path/to/your/image.jpg")
    
    img = np.copy(results[0].orig_img)
    b_mask = np.zeros(img.shape[:2], np.uint8)
    contour = results[0].masks.xy[0].astype(np.int32).reshape(-1, 1, 2)
    cv2.drawContours(b_mask, [contour], -1, (255, 255, 255), cv2.FILLED)
    
    mask3ch = cv2.cvtColor(b_mask, cv2.COLOR_GRAY2BGR)
    isolated = cv2.bitwise_and(mask3ch, img)

    完整的逐检测循环请参阅完整示例。

  • 主要有两种方式。对于黑色背景,将掩码转换为三个通道并使用 cv2.bitwise_and()。对于透明背景(保存为 PNG 时),将掩码作为第四个 alpha 通道叠加,并使用 np.dstack([img, b_mask])。两种方式都在隔离对象一节中介绍。

  • 从检测结果中读取边界框坐标,并对隔离后的图像进行切片:

    x1, y1, x2, y2 = results[0].boxes.xyxy[0].cpu().numpy().astype(np.int32)
    iso_crop = isolated[y1:y2, x1:x2]

    在 Predict 模式文档中了解更多边界框结果信息。

  • Ultralytics YOLO 可实现快速、实时的实例分割,准确生成掩码和边界框,并提供简单的 Python API,只需几行 OpenCV 代码即可从推理结果中提取独立目标。

  • 可以。在 predict() 中使用 save_crop 参数,即可保存保留原始背景的边界框裁剪图:

    results = model.predict(source="path/to/your/image.jpg", save_crop=True)

    更多信息请参阅预测模式推理参数部分。

评论16

提出问题、分享结果,或帮助其他人。

  1. AbdulGhofur23GitHub

    How can I extract a mask from a TIFF (Tag Image File Format) so that the mask's coordinates align with the projection of the TIFF image?

    1. Glenn JocherUltralytics

      To extract masks from a TIFF image while aligning the mask's coordinates with the image's projection, you can use the segmentation results from Ultralytics YOLO. Generate the mask using results.masks.xy for contour points and overlay it on the TIFF image. Ensure alignment by maintaining the original image dimensions during processing. For more, refer to the Isolating Segmentation Objects Guide. Let us know if you need further assistance!

      1
  2. vedhanayagan9626GitHub

    I used yolo v11n seg model , i got a segmented image as output with bounding boxes in those some of predictions are overlapped like a bounding box on two items as single so i try to remove a rectangle and associated segmentation from an image and add new rectangle by hold and drag functionality its looks likw working but i cant get optimized result. Is there any way to remove and add new bounding boxes after a segmentation result?

    1. Glenn JocherUltralytics

      @vedhanayagan9626 to address overlapping predictions, you can manually refine results by manipulating masks and bounding boxes post-inference. Use the mask and bounding box coordinates from results[0].masks and results[0].boxes to remove or modify specific predictions. For new bounding boxes, you might need to create and integrate them into your results structure manually. Refer to the Isolating Segmentation Objects guide for detailed steps. For optimized results, ensure adequate data quality and model tuning.

  3. SerhiiPostupaievGitHub

    Hello, can you please explain how the YOLOv8 performs upscaling? For example, my input images are 854x480, YOLOv8 scales them up to 864x512. How does it exactly performs upscaling? Does it use padding or some other approaches?

    1. Paula DerrengerUltralytics

      @SerhiiPostupaiev hello! In YOLOv8, images are scaled up or down to a specific model input size, optimally a square format (like 640x640) or a dimension supported by the model (such as 864x512 in your case). The upscaling for images that do not exactly match the input dimensions typically involves:

      1. Resizing the image to the closest dimension that maintains the aspect ratio.
      2. Padding the resized image to reach the required input size, ensuring that the aspect ratio is preserved.

      For instance, for an image of 854x480, YOLOv8 may:

      • Resize the image to fit the model's width of 864 while maintaining the aspect ratio, which adjusts the height accordingly.
      • Pad the height to the nearest model-supported dimension (like 512), adding pixel values (usually zeros) to the top and bottom or sides of the image to reach the required size.

      Here’s a rough example in Python using OpenCV for how padding might be handled:

      import cv2
      import numpy as np
      
      # Example image load
      image = cv2.imread('input.jpg')
      # Resizing the image
      resized_image = cv2.resize(image, (864, keep_aspect_ratio_height))  # Example variable for illustration
      
      # Padding the resized image
      top_padding = (512 - resized_image.shape[0]) // 2
      bottom_padding = 512 - resized_image.shape[0] - top_padding
      padded_image = cv2.copyMakeBorder(resized_image, top_padding, bottom_padding, 0, 0, cv2.BORDER_CONSTANT, value=[0, 0, 0])

      This ensures the entire image is used during prediction without distorting its original aspect ratio. 😊

  4. EnricoMendezGitHub

    Hello, I run this code example but it only display one object's mask, is it posible to join all the objects' masks in a single frame?

    1. Glenn JocherUltralytics

      Hey there! 😊 Absolutely, you can combine all the objects' masks into a single frame. Just modify the section where you're drawing the contours onto the mask. Instead of creating a new binary mask b_mask for each object, you initialize one common mask before iterating through the objects and draw every object's contour on this common mask. Here's a quick example to illustrate:

      # Initialize one common binary mask for all objects
      b_mask_all = np.zeros(img.shape[:2], np.uint8)
      
      for ci, c in enumerate(r):
          # Your contour extraction remains the same
          contour = c.masks.xy.pop().astype(np.int32).reshape(-1, 1, 2)
          # Draw each contour onto the common mask
          cv2.drawContours(b_mask_all, [contour], -1, (255, 255, 255), cv2.FILLED)
      
      # After the loop, you can use `b_mask_all` which contains all objects' masks

      Just keep in mind, with b_mask_all, you'll have all the detected objects' masks on one mask. If the objects overlap in your image, their masks will also overlap in b_mask_all. Hope this helps! Let us know if you have more questions. 😊

  5. mariamg03GitHub

    hi is there a way I can apply the edits on the binary image then revert back to the original with the mask part edited?

    1. Paula DerrengerUltralytics

      Hey there! 👋 Yes, you can certainly make edits on a binary mask and then apply those changes back to the original image, essentially modifying only the parts that the mask covers. Here's a quick outline of how you could achieve this with an example focusing on segmentation tasks:

      1. After segmenting your image and obtaining the binary mask (let's say b_mask), you can perform any edits you like on this mask. For example, inverting the mask, dilating, or eroding.

      2. To apply the edited mask back to the original image (orig_img), you would use the mask to isolate the area of interest in the original image and then apply your edits.

      Here's a code snippet demonstrating this with a simple inversion edit:

      import cv2
      import numpy as np
      
      # Assuming 'b_mask' is your binary mask and 'orig_img' is your original image
      # Let's say you inverted the binary mask
      edited_mask = 255 - b_mask  # Inverting mask
      
      # Convert the single channel binary mask to a 3 channel mask to match the original image
      edited_mask_3ch = cv2.cvtColor(edited_mask, cv2.COLOR_GRAY2BGR)
      
      # Apply the edited mask back to the original image
      # This line performs bitwise-and operation to only keep parts of the image where the mask is white
      edited_image = cv2.bitwise_and(orig_img, edited_mask_3ch)
      
      # At this point, 'edited_image' contains the original image with edits applied to the segmented part

      This is a simplified example to show the concept. Depending on the edits you've made to b_mask, you might need to adjust how you apply edited_mask back to orig_img.

      Remember, after editing the mask, you can always revert it back to its original form if you've kept a copy before editing, or you can apply the inverse of your edits.

      I hope this helps! Let me know if you have more questions. Happy coding! 😊

  6. SafiislamianGitHub

    Hi sir, thanku for your response does load dataset will work or not. As the dataset is in yolo form where we have different folders and subfolders like train having images and labels etc

    On Wed, Mar 27, 2024, 7:03 AM Glenn Jocher @.***> wrote:

    Hey there! 👋 Glad to see you're interested in applying YOLOv8 for instance segmentation with a dataset from Roboflow. Below is a quick guide to help you get started with applying YOLOv8 for detection and then using masks for the dataset based on class. Step 1: Install Ultralytics YOLO

    First, ensure you have Ultralytics installed:

    pip install ultralytics

    Step 2: Load Your Model

    from ultralytics import YOLOmodel = YOLO('yolov8n-seg.pt') # Use yolov8n-seg.pt for segmentation models

    Step 3: Predict and Isolate Objects

    Now, for predicting and isolating objects based on their classes, you'll dive into the prediction results to access and manipulate the segmentation masks:

    results = model.predict(source='path/to/your/dataset')for result in results: for detection in result: class_name = detection.names[detection.boxes.cls.tolist().pop()] # Create a binary mask from detection binary_mask = np.zeros(result.orig_img.shape[:2], np.uint8) contour = detection.masks.xy.pop().astype(np.int32).reshape(-1, 1, 2) cv2.drawContours(binary_mask, [contour], -1, (255, 255, 255), cv2.FILLED) # Example to save or further process the mask

    Note: Replace 'path/to/your/dataset' with the actual path to your images. Step 4: Saving or Processing the Masks

    You may wish to save each mask based on its class for processing later. You can modify the loop to save each binary mask to a file using cv2.imwrite() method.

    cv2.imwrite(f'{class_name}_mask.jpg', binary_mask)

    This will save each mask with its respective class name as part of the filename, allowing for easy organization by class.

    Hope this helps! 😊 Let us know if you need further assistance or have more questions. Happy coding!

    — Reply to this email directly, view it on GitHub https://github.com/orgs/ultralytics/discussions/9334#discussioncomment-8922405, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUP75TG3RFWAL7YRMWDR66DY2ILE5AVCNFSM6AAAAABFJ334B2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DSMRSGQYDK . You are receiving this because you commented.Message ID: @.*** com>

    1. Paula DerrengerUltralytics(已编辑)

      Hi there! 😄

      Absolutely, loading a dataset structured in YOLO format (with separate folders for train/test images and labels) works well with YOLO26. You can specify the dataset structure in a YAML file and provide this file when initiating training or validation processes. Here's an example data.yaml file structure for your dataset:

      train: /path/to/dataset/train/images  #  path to images folder
      val: /path/to/dataset/val/images      #  path to validation images
      
      # Classes
      nc: 80  # number of classes
      names: ['class1', 'class2', ...]  # class names

      And here's how you might start training with your dataset:

      yolo detect train data=data.yaml model=yolo26n.pt

      Make sure to replace /path/to/dataset/train/images and /path/to/dataset/val/images with the actual paths to your dataset and similarly update the nc (number of classes) and names (class labels) accordingly.

      If you have any more questions or need further assistance, feel free to ask. Happy to help! 🚀

  7. SafiislamianGitHub

    There is a dataset on roboflow for instance segmentation, i want to apply first yolov8 and then masks for all the dataset arranged on the basis of their classes. please provide me details and code for that Thank you...

    1. Glenn JocherUltralytics(已编辑)

      Hey there! 👋 Glad to see you're interested in applying YOLO26 for instance segmentation with a dataset from Roboflow. Below is a quick guide to help you get started with applying YOLO26 for detection and then using masks for the dataset based on class.

      Step 1: Install Ultralytics YOLO

      First, ensure you have Ultralytics installed:

      pip install ultralytics
      Step 2: Load Your Model
      from ultralytics import YOLO
      model = YOLO('yolo26n-seg.pt')  # Use yolo26n-seg.pt for segmentation models
      Step 3: Predict and Isolate Objects

      Now, for predicting and isolating objects based on their classes, you'll dive into the prediction results to access and manipulate the segmentation masks:

      results = model.predict(source='path/to/your/dataset')
      for result in results:
          for detection in result:
              class_name = detection.names[detection.boxes.cls.tolist().pop()]
              # Create a binary mask from detection
              binary_mask = np.zeros(result.orig_img.shape[:2], np.uint8)
              contour = detection.masks.xy.pop().astype(np.int32).reshape(-1, 1, 2)
              cv2.drawContours(binary_mask, [contour], -1, (255, 255, 255), cv2.FILLED)
              # Example to save or further process the mask

      Note: Replace 'path/to/your/dataset' with the actual path to your images.

      Step 4: Saving or Processing the Masks

      You may wish to save each mask based on its class for processing later. You can modify the loop to save each binary mask to a file using cv2.imwrite() method.

      cv2.imwrite(f'{class_name}_mask.jpg', binary_mask)

      This will save each mask with its respective class name as part of the filename, allowing for easy organization by class.

      Hope this helps! 😊 Let us know if you need further assistance or have more questions. Happy coding!