Instance Segmentation Datasets Overview

Instance segmentation is a computer vision task that involves identifying and delineating individual objects within an image. This guide provides an overview of dataset formats supported by Ultralytics YOLO for instance segmentation tasks, along with instructions on how to prepare, convert, and use these datasets for training your models.

Supported Dataset Formats#

Ultralytics YOLO format#

The dataset label format used for training YOLO segmentation models is as follows:

1. One text file per image: Each image in the dataset has a corresponding text file with the same name as the image file and the ".txt" extension.
2. One row per object: Each row in the text file corresponds to one object instance in the image.
3. Object information per row: Each row contains the following information about the object instance:
   • Object class index: An integer representing the class of the object (e.g., 0 for person, 1 for car, etc.).
   • Object polygon coordinates: The (x, y) points outlining the object's mask, normalized to be between 0 and 1.

The format for a single row in the segmentation dataset file is as follows:

<class-index> <x1> <y1> <x2> <y2> ... <xn> <yn>

In this format, <class-index> is the index of the class for the object, and <x1> <y1> <x2> <y2> ... <xn> <yn> are the normalized polygon coordinates of the object's segmentation mask (values are in [0, 1] relative to image width and height). The coordinates are separated by spaces.

Here is an example of the YOLO dataset format for a single image with two objects made up of a 3-point segment and a 5-point segment.

0 0.681 0.485 0.670 0.487 0.676 0.487
1 0.504 0.000 0.501 0.004 0.498 0.004 0.493 0.010 0.492 0.0104

Dataset YAML format#

The Ultralytics framework uses a YAML file format to define the dataset and model configuration for training segmentation models. Here is an example of the YAML format used for defining a segmentation dataset:

# Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license

# COCO8-seg dataset (first 8 images from COCO train2017) by Ultralytics
# Documentation: https://docs.ultralytics.com/datasets/segment/coco8-seg
# Example usage: yolo train data=coco8-seg.yaml
# parent
# ├── ultralytics
# └── datasets
#     └── coco8-seg ← downloads here (1 MB)

# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: coco8-seg # dataset root dir
train: images/train # train images (relative to 'path') 4 images
val: images/val # val images (relative to 'path') 4 images
test: # test images (optional)

# Classes
names:
  0: person
  1: bicycle
  2: car
  3: motorcycle
  4: airplane
  5: bus
  6: train
  7: truck
  8: boat
  9: traffic light
  10: fire hydrant
  11: stop sign
  12: parking meter
  13: bench
  14: bird
  15: cat
  16: dog
  17: horse
  18: sheep
  19: cow
  20: elephant
  21: bear
  22: zebra
  23: giraffe
  24: backpack
  25: umbrella
  26: handbag
  27: tie
  28: suitcase
  29: frisbee
  30: skis
  31: snowboard
  32: sports ball
  33: kite
  34: baseball bat
  35: baseball glove
  36: skateboard
  37: surfboard
  38: tennis racket
  39: bottle
  40: wine glass
  41: cup
  42: fork
  43: knife
  44: spoon
  45: bowl
  46: banana
  47: apple
  48: sandwich
  49: orange
  50: broccoli
  51: carrot
  52: hot dog
  53: pizza
  54: donut
  55: cake
  56: chair
  57: couch
  58: potted plant
  59: bed
  60: dining table
  61: toilet
  62: tv
  63: laptop
  64: mouse
  65: remote
  66: keyboard
  67: cell phone
  68: microwave
  69: oven
  70: toaster
  71: sink
  72: refrigerator
  73: book
  74: clock
  75: vase
  76: scissors
  77: teddy bear
  78: hair drier
  79: toothbrush

# Download script/URL (optional)
download: https://github.com/ultralytics/assets/releases/download/v0.0.0/coco8-seg.zip

The train, val, and test fields point to the training, validation, and test images. Each accepts a directory, an image file, a list of directories or image files, or a UTF-8 *.txt file listing one image path per line (paths starting with ./ resolve relative to the *.txt file). A *.txt file is useful to train on a subset of a directory, skip unlabeled images, or combine images from multiple sources into one split.

path: datasets/coco8-seg # dataset root
train: train.txt # a directory, an image, a list e.g. [images/a, images/b], or a *.txt file
val: val.txt
names:
  0: person

names is a dictionary of class names. The order of the names should match the order of the object class indices in the YOLO dataset files.

Usage#

from ultralytics import YOLO

# Load a model
model = YOLO("yolo26n-seg.pt")  # load a pretrained model (recommended for training)

# Train the model
results = model.train(data="coco8-seg.yaml", epochs=100, imgsz=640)

Supported Datasets#

Ultralytics YOLO supports various datasets for instance segmentation tasks. Here's a list of the most commonly used ones. Most of these datasets are also hosted on Ultralytics Platform, where you can browse the images and annotations, view dataset statistics, and clone them for cloud training.

• Carparts-seg: A specialized dataset focused on the segmentation of car parts, ideal for automotive applications. It includes a variety of vehicles with detailed annotations of individual car components.
• COCO: A comprehensive dataset for object detection, segmentation, and captioning, featuring over 200K labeled images across a wide range of categories.
• COCO8-seg: A compact, 8-image subset of COCO designed for quick testing of segmentation model training, ideal for CI checks and workflow validation in the ultralytics repository.
• COCO128-seg: A smaller dataset for instance segmentation tasks, containing a subset of 128 COCO images with segmentation annotations.
• Crack-seg: A dataset tailored for the segmentation of cracks in various surfaces. Essential for infrastructure maintenance and quality control, it provides detailed imagery for training models to identify structural weaknesses.
• Package-seg: A dataset dedicated to the segmentation of different types of packaging materials and shapes. It's particularly useful for logistics and warehouse automation, aiding in the development of systems for package handling and sorting.

Adding your own dataset#

If you have your own dataset and would like to use it for training segmentation models with Ultralytics YOLO format, ensure that it follows the format specified above under "Ultralytics YOLO format". Convert your annotations to the required format and specify the paths, number of classes, and class names in the YAML configuration file. Keep images/ and labels/ as separate folders at the same level, with matching subfolder structure; placing label .txt files in the image folder can cause the model to miss labels.

Port or Convert Label Formats#

COCO Dataset Format to YOLO Format#

You can convert labels from the popular COCO dataset format to the YOLO format using the following code snippet:

from ultralytics.data.converter import convert_coco

convert_coco(labels_dir="path/to/coco/annotations/", use_segments=True)

This conversion tool can be used to convert the COCO dataset or any dataset in the COCO format to the Ultralytics YOLO format.

Remember to double-check if the dataset you want to use is compatible with your model and follows the necessary format conventions. Properly formatted datasets are crucial for training successful segmentation models.

Auto-Annotation#

Auto-annotation generates a segmentation dataset from a pretrained detection model, so you can label a large number of images without drawing every mask by hand. To label in the browser instead, Ultralytics Platform offers SAM, YOLO, and class-prompted smart annotation, and the VLM leaderboard compares the prompted models on detection accuracy, cost, and speed.

Generate Segmentation Dataset Using a Detection Model#

To auto-annotate your dataset using the Ultralytics framework, you can use the auto_annotate function as shown below:

from ultralytics.data.annotator import auto_annotate

auto_annotate(data="path/to/images", det_model="yolo26x.pt", sam_model="sam_b.pt")

The auto_annotate function takes the path to your images, along with optional arguments for specifying the pretrained detection models, e.g. YOLO26, YOLO11, or other models, and segmentation models, e.g. SAM, SAM 2, MobileSAM, or SAM 3, the device to run the models on, and the output directory for saving the annotated results.

Using pretrained models, auto-annotation significantly reduces the time and effort required to create high-quality segmentation datasets. This is particularly useful for researchers and developers working with large image collections, as it allows them to focus on model development and evaluation rather than manual annotation.

Visualize Dataset Annotations#

Before training your model, it's often helpful to visualize your dataset annotations to ensure they're correct. Ultralytics provides a utility function for this purpose. It reads detection-format box labels (<class-index> <x_center> <y_center> <width> <height>) only, so it cannot parse segmentation polygon label files:

from ultralytics.data.utils import visualize_image_annotations

label_map = {  # Define the label map with all annotated class labels.
    0: "person",
    1: "car",
}

# Visualize
visualize_image_annotations(
    "path/to/image.jpg",  # Input image path.
    "path/to/annotations.txt",  # Annotation file path for the image.
    label_map,
)

This function draws bounding boxes, labels objects with class names, and adjusts text color for better readability, helping you identify and correct any annotation errors before training.

Converting Segmentation Masks to YOLO Format#

If you have grayscale mask images where each pixel value is the class index + 1 (0 is background), you can convert them to the YOLO segmentation format using:

from ultralytics.data.converter import convert_segment_masks_to_yolo_seg

# For datasets like COCO with 80 classes
convert_segment_masks_to_yolo_seg(masks_dir="path/to/masks_dir", output_dir="path/to/output_dir", classes=80)

This utility converts the mask images into the YOLO segmentation format and saves them in the specified output directory.

FAQ#