开始使用

姿态估计数据集概览

支持的数据集格式#

Ultralytics YOLO 格式#

用于训练 YOLO 姿态模型的数据集标签格式如下:

  1. 每张图像对应一个文本文件:数据集中的每张图像都有一个对应的文本文件,文件名与图像文件相同,扩展名为“.txt”。
  2. 每个物体占一行:文本文件中的每一行对应图像中的一个物体实例。
  3. 每行包含物体信息:每一行包含有关该物体实例的以下信息:
    • 物体类别索引:表示物体类别的整数(例如,0 表示 person,1 表示 car,依此类推)。
    • 对象中心坐标:对象中心的 x 和 y 坐标,归一化到 0 到 1 之间。
    • 对象宽度和高度:对象的宽度和高度,归一化到 0 到 1 之间。
    • 对象关键点坐标:对象的关键点坐标,归一化到 0 到 1 之间。

以下是姿态估计任务的标签格式示例:

2D 关键点格式

<class-index> <x> <y> <width> <height> <px1> <py1> <px2> <py2> ... <pxn> <pyn>

带关键点可见性信息的格式(包含每个点的可见性)

<class-index> <x> <y> <width> <height> <px1> <py1> <p1-visibility> <px2> <py2> <p2-visibility> ... <pxn> <pyn> <pn-visibility>

在这种格式中,<class-index> 是物体的类别索引,<x> <y> <width> <height> 是边界框的归一化坐标,<px1> <py1> <px2> <py2> ... <pxn> <pyn> 是关键点的归一化坐标。可见性通道是可选的,但对标注遮挡情况的数据集很有用。

数据集 YAML 格式#

Ultralytics 框架使用 YAML 文件格式定义数据集和模型配置,以训练姿态估计模型。以下是用于定义姿态数据集的 YAML 格式示例:

ultralytics/cfg/datasets/coco8-pose.yaml
# Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license

# COCO8-pose dataset (first 8 images from COCO train2017) by Ultralytics
# Documentation: https://docs.ultralytics.com/datasets/pose/coco8-pose
# Example usage: yolo train data=coco8-pose.yaml
# parent
# ├── ultralytics
# └── datasets
#     └── coco8-pose ← downloads here (1 MB)

# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: coco8-pose # dataset root dir
train: images/train # train images (relative to 'path') 4 images
val: images/val # val images (relative to 'path') 4 images
test: # test images (optional)

# Keypoints
kpt_shape: [17, 3] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible)
flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]

# Classes
names:
  0: person

# Keypoint names per class
kpt_names:
  0:
    - nose
    - left_eye
    - right_eye
    - left_ear
    - right_ear
    - left_shoulder
    - right_shoulder
    - left_elbow
    - right_elbow
    - left_wrist
    - right_wrist
    - left_hip
    - right_hip
    - left_knee
    - right_knee
    - left_ankle
    - right_ankle

# Download script/URL (optional)
download: https://github.com/ultralytics/assets/releases/download/v0.0.0/coco8-pose.zip

train、val 和 test 字段分别指向训练、验证和测试图像。每个字段均接受目录、图像文件、目录或图像文件列表,或 UTF-8 *.txt 文件;该文件每行列出一个图像路径(以 ./ 开头的路径将相对于 *.txt 文件解析)。使用 *.txt 文件可以基于目录的子集进行训练、跳过未标注的图像,或将来自多个来源的图像合并到一个数据集划分中。

将图像路径写入 `*.txt` 文件
path: datasets/coco8-pose # dataset root
train: train.txt # a directory, an image, a list e.g. [images/a, images/b], or a *.txt file
val: val.txt
kpt_shape: [17, 3] # required for pose datasets
names:
  0: person

names 是一个类别名称字典。名称的顺序应与 YOLO 数据集文件中对象类别索引的顺序一致。

(可选)flip_idx 将每个关键点映射到其镜像位置,使水平翻转增强在人体或面部等对称骨架上保持左右一致。对于按 [左眼、右眼、鼻子、左嘴角、右嘴角] = [0, 1, 2, 3, 4] 编号的五个面部关键点,flip_idx 为 [1, 0, 2, 4, 3]:左右成对的 0-1 和 3-4 会互换,鼻子则保持原索引。

(可选)kpt_oks_sigmas 设置训练和验证时使用的自定义关键点 OKS sigma,例如 [0.26, 0.25, 0.25, ...]。列表长度必须等于 kpt_shape 中的关键点数量 N,且每个值都必须为正数。若未设置,对于 kpt_shape: [17, 3],将使用 COCO 的 17 个关键点 sigma;其他情况下则使用统一的 1/N。

用法#

示例
from ultralytics import YOLO

# 加载模型
model = YOLO("yolo26n-pose.pt")  # 加载预训练模型(推荐用于训练)

# 训练模型
results = model.train(data="coco8-pose.yaml", epochs=100, imgsz=640)

支持的数据集#

本节介绍与 Ultralytics YOLO 格式兼容、可用于训练姿态估计模型的数据集。其中大多数数据集也托管在 Ultralytics Platform 上,你可以浏览图像和标注、查看数据集统计信息,并克隆数据集以进行云端训练。

COCO-Pose#

  • 描述:COCO-Pose 是一个大规模人体姿态估计数据集,涵盖 COCO 2017 图像中包含关键点标注人物的图像。
  • 标签格式:与上文所述的 Ultralytics YOLO 格式相同,包含人体姿态关键点。
  • 类别数量:1(person)。
  • 关键点:包含 17 种关键点类型,包括鼻子、眼睛、耳朵、肩膀、肘部、手腕、臀部、膝盖和脚踝,每个关键点都带有一个可见性维度。
  • 用途:适合用于训练人体姿态估计模型。
  • 补充说明:该数据集基于 COCO Keypoints 2017 挑战赛构建,包含 58,945 张图像,标注了 156,165 个人物。
  • 了解 COCO-Pose 的更多信息

COCO8-Pose#

  • 描述:Ultralytics COCO8-Pose 是一个小型姿态估计数据集,由 COCO train 2017 集的前 8 张图像组成,其中 4 张用于训练,4 张用于验证。
  • 标签格式:与上文所述的 Ultralytics YOLO 格式相同,包含人体姿态关键点。
  • 类别数量:1(person)。
  • 关键点:包含 17 种关键点类型,包括鼻子、眼睛、耳朵、肩膀、肘部、手腕、臀部、膝盖和脚踝,每个关键点都带有一个可见性维度。
  • 用途:适合用于测试和调试姿态估计模型,或试验新的关键点检测方法。
  • 补充说明:COCO8-Pose 非常适合进行合理性检查和 CI 检查。
  • 了解 COCO8-Pose 的更多信息

Dog-Pose#

  • 描述:Ultralytics Dog-Pose 数据集包含 6,773 张训练图像和 1,703 张验证图像,用于犬类关键点估计。
  • 标签格式:遵循 Ultralytics YOLO 格式,标注了适用于狗的解剖结构的多个关键点。
  • 类别数量:1(dog)。
  • 关键点:包含 24 个关键点,每个关键点都带有可见性维度,针对狗的姿态进行了设计,例如四肢、关节和头部位置。
  • 用途:非常适合训练模型在各种场景中估计狗的姿态,应用范围涵盖研究和现实场景。
  • 补充说明:源图像来自 Stanford Dogs Dataset。
  • 了解 Dog-Pose 的更多信息

手部关键点#

  • 描述:Ultralytics Hand Keypoints 数据集包含 26,768 张图像,其中 18,776 张用于训练,7,992 张用于验证。
  • 标签格式:与上文所述的 Ultralytics YOLO 格式相同,但包含人手的 21 个关键点和一个可见性维度。
  • 类别数量:1(hand)。
  • 关键点:21 个关键点。
  • 用途:非常适合人体手部姿态估计和手势识别。
  • 补充说明:关键点标注使用 Google MediaPipe 生成,以确保标注一致。
  • 了解手部关键点的更多信息

Tiger-Pose#

  • 描述:Ultralytics Tiger-Pose 数据集包含 263 张图像,图像取自一个 YouTube 视频,其中 210 张用于训练,53 张用于验证。
  • 标签格式:与上文所述的 Ultralytics YOLO 格式相同,包含用于动物姿态的 12 个关键点,不含可见性维度。
  • 类别数量:1(tiger)。
  • 关键点:12 个关键点。
  • 用途:适用于动物姿态估计或任何非人体姿态估计任务。
  • 补充说明:根据 AGPL-3.0 许可证发布。
  • 了解 Tiger-Pose 的更多信息

添加你自己的数据集#

如果你有自己的数据集,并希望使用 Ultralytics YOLO 格式训练姿态估计模型,请确保数据集遵循上文“Ultralytics YOLO 格式”中指定的格式。将标注转换为所需格式,并在 YAML 配置文件中指定路径、类别数量和类别名称。

如果想完全跳过转换步骤,Ultralytics Platform 支持你上传原始图像、在浏览器中标注关键点,并直接使用生成的数据集进行训练。

转换工具#

Ultralytics 提供转换工具,可将常用的 COCO 数据集格式标签转换为 YOLO 格式:

示例
from ultralytics.data.converter import convert_coco

convert_coco(labels_dir="path/to/coco/annotations/", use_keypoints=True)

此转换工具可用于将 COCO 数据集或任何采用 COCO 格式的数据集转换为 Ultralytics YOLO 格式。use_keypoints 参数用于指定是否在转换后的标签中包含关键点(用于姿态估计)。

常见问题#

  • 姿态估计数据集的 Ultralytics YOLO 格式要求为每张图像创建一个对应的文本文件作为标签。文本文件的每一行都存储一个对象实例的信息:

    • 对象类别索引
    • 对象中心坐标(归一化的 x 和 y)
    • 对象宽度和高度(归一化)
    • 对象关键点坐标(归一化的 pxn 和 pyn)

    对于 2D 姿态,关键点包含归一化的 x 和 y 坐标。添加可见性维度后,每个关键点还带有一个可见性标志。详情请参阅 Ultralytics YOLO 格式。

  • coco-pose.yaml 随软件包一起提供,并会在首次使用时下载图像和标签,因此无需手动准备:

    from ultralytics import YOLO
    
    model = YOLO("yolo26n-pose.pt")  # 加载预训练模型
    results = model.train(data="coco-pose.yaml", epochs=100, imgsz=640)

    数据集详情请参阅 COCO-Pose,完整参数列表请参阅 训练页面。

  • 添加数据集的步骤:

    1. 将标注转换为 Ultralytics YOLO 格式。

    2. 创建 YAML 配置文件,指定数据集路径、类别数量和类别名称。

    3. 使用配置文件训练模型:

      from ultralytics import YOLO
      
      model = YOLO("yolo26n-pose.pt")
      results = model.train(data="your-dataset.yaml", epochs=100, imgsz=640)

      完整步骤请参阅添加自己的数据集部分。

  • Ultralytics YOLO 中的数据集 YAML 文件用于定义训练所需的数据集和模型配置。文件会指定训练、验证和测试图像的路径、关键点形状、类别名称以及其他配置选项。这种结构化格式有助于简化数据集管理和模型训练。以下是 YAML 格式示例:

    # Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license
    
    # COCO8-pose dataset (first 8 images from COCO train2017) by Ultralytics
    # Documentation: https://docs.ultralytics.com/datasets/pose/coco8-pose
    # Example usage: yolo train data=coco8-pose.yaml
    # parent
    # ├── ultralytics
    # └── datasets
    #     └── coco8-pose ← downloads here (1 MB)
    
    # Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
    path: coco8-pose # dataset root dir
    train: images/train # train images (relative to 'path') 4 images
    val: images/val # val images (relative to 'path') 4 images
    test: # test images (optional)
    
    # Keypoints
    kpt_shape: [17, 3] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible)
    flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
    
    # Classes
    names:
      0: person
    
    # Keypoint names per class
    kpt_names:
      0:
        - nose
        - left_eye
        - right_eye
        - left_ear
        - right_ear
        - left_shoulder
        - right_shoulder
        - left_elbow
        - right_elbow
        - left_wrist
        - right_wrist
        - left_hip
        - right_hip
        - left_knee
        - right_knee
        - left_ankle
        - right_ankle
    
    # Download script/URL (optional)
    download: https://github.com/ultralytics/assets/releases/download/v0.0.0/coco8-pose.zip

    了解如何创建 YAML 配置文件,请参阅数据集 YAML 格式。

  • Ultralytics 提供了一个转换工具,可将 COCO 数据集标签转换为 YOLO 格式,包括关键点信息:

    from ultralytics.data.converter import convert_coco
    
    convert_coco(labels_dir="path/to/coco/annotations/", use_keypoints=True)

    这个工具可以将 COCO 格式的数据集导入 YOLO 项目。

评论69

提出问题、分享结果,或帮助其他人。

  1. OfekDayanGitHub

    I trained pose models (n - x) on a custom data - pitch on a football playground (skeleton with 32 points). Most of the key-points are detected on the correct location, but they have wrong ids.

    What could be the reason?

  2. UltralyticsAssistantGitHubUltralytics

    👋 Hello, thank you for your interest in Ultralytics 🚀! We recommend checking out the Docs for detailed information about pose estimation datasets. Specifically, you can explore the Pose Datasets section, where you'll find guidance on supported formats like COCO-Pose, COCO8-Pose, Tiger-Pose, and instructions for adding custom datasets.

    If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug the issue.

    For custom training ❓ Questions, please include as much detail as possible, such as dataset examples, training logs, or specific challenges. Ensure you're following our Tips for Best Training Results for optimal performance.

    Join the Ultralytics community where it suits you best:

    • For real-time discussions, visit Discord 🎧.
    • Explore in-depth topics on Discourse.
    • Share insights and connect on our Subreddit.

    Upgrade

    To ensure you are using the latest features and fixes, upgrade to the latest ultralytics package with:

    pip install -U ultralytics

    Verify your environment includes all requirements and is running Python>=3.8 and PyTorch>=1.8.

    Environments

    Run YOLO in any of these up-to-date verified environments with preinstalled dependencies:

    Status

    Ultralytics CI

    If this badge is green, all Ultralytics CI tests are passing. CI tests validate the functionality of YOLO Modes and Tasks across macOS, Windows, and Ubuntu.

    This is an automated response, but an Ultralytics engineer will assist you shortly. Let us know if you need further guidance! 😊

    1. Glenn JocherUltralytics

      @Alebep Currently in the Ultralytics YOLO framework, all classes within a single dataset must use the same number of keypoints as defined by the kpt_shape parameter in your YAML file. This is a framework-level constraint where the model architecture expects a consistent keypoint structure across all detected objects.

      If you need to work with objects requiring different numbers of keypoints (like your dog with 5 and turtle with 3 example), you would need to:

      1. Train separate models for each keypoint configuration using different datasets
      2. Or use a unified keypoint count approach where you define all possible keypoints and mark unused ones as invisible for certain classes

      This is because the pose estimation head in YOLO is structured to predict a fixed number of keypoints for all objects. Each dataset's configuration defines this through the kpt_shape parameter, which applies uniformly to all classes in that dataset.

      For examples of different keypoint configurations, you can look at our existing pose datasets like Dog-Pose (24 keypoints) and Tiger-Pose (12 keypoints), each optimized for their specific subject.

  3. AlebepGitHub

    Can the number of keypoints vary by class? For example, could the 'dog' class have 5 keypoints while the 'turtle' class has only 3? Or do all classes need to have the same number of keypoints?

    1. Glenn JocherUltralytics

      Hi there! 👋 In Ultralytics YOLO pose estimation, all classes in a single dataset must share the same number of keypoints as defined by the kpt_shape parameter in the dataset YAML file. This configuration applies uniformly across all classes within that dataset. For mixed keypoint requirements across object types, you'll need to train separate models on different datasets. Check out our Pose Estimation Datasets Guide for details on configuring keypoint dimensions.

      1
  4. foolishcqxGitHub

    首先,非常感谢你们的工作!最近我在进行自定义的关键点检测时遇到了一个问题,我们的模型在训练集和验证集上的效果在训练过程中指标都表现的很好,但是在我测试效果很糟糕。是否有什么技巧可以改善这个问题? 数据标签格式: 0 0.528905 0.425485 0.501863 0.742139 0.385417 0.384615 0.406250 0.377622 0.429688 0.374126 0.450521 0.384615 0.427083 0.391608 0.403646 0.395105 0.565104 0.388112 0.585938 0.377622 0.609375 0.381119 0.630208 0.391608 0.609375 0.398601 0.585938 0.395105 0.450521 0.636364 0.486979 0.632867 0.510417 0.636364 0.531250 0.632867 0.565104 0.632867 0.531250 0.632867 0.510417 0.636364 0.486979 0.632867 配置文件的格式:

    path: /data/zhongjiafeng/ultralytics-main/posedataset # dataset root dir train: /data/zhongjiafeng/ultralytics-main/posedataset/train/images # train images (relative to 'path') 4 images val: /data/zhongjiafeng/ultralytics-main/posedataset/val/images # val images (relative to 'path') 4 images test: # test images (optional)

    Keypoints

    kpt_shape: [20, 2] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible)

    Classes

    names: 0: face

    我们是在加载yolo11s-pose.pt模型的基础上进行训练的

    1. Glenn JocherUltralytics

      @foolishcqx 感谢您的详细描述!从您的配置来看,关键点维度设置为2但缺少必要的flip_idx对称翻转索引,这会影响数据增强效果。建议参考COCO-Pose的YAML配置,为面部对称关键点(如双眼/嘴角)添加正确的flip_idx,并验证关键点坐标是否已归一化到0-1范围。具体可查看Pose数据集格式文档中的对称点配置说明。

  5. matteobonaGitHub

    Hi, I was trying to train a custom pose model on a dataset with 4 classes. The first class has 31 keypoints, while the others do not have keypoints. I created my YAML file as follows:

    train: "images\train" # train images val: "images\valid" #val images kpt_shape: [31, 3] nc: 4 names: ['pitch', 'player', 'referee', 'ball']

    And this is an example of a .txt file of annotations: 0 0.5002604166666667 0.5421296296296296 0.9994791666666667 0.8953703703703704 0.367078125 0.12500925925925926 2 0.2983958333333333 0.1687037037037037 2 0.22916145833333335 0.2129722222222222 2 0.0834375 0.3101851851851852 2 0 0 0 0 0 0 0.283984375 0.22016666666666668 2 0.14989583333333334 0.32462962962962966 2 0.463375 0.18461111111111111 2 0.39940625 0.2422314814814815 2 0.30180208333333336 0.3329259259259259 2 0.1875 0.4439814814814815 2 0.89265625 0.18564814814814815 2 0.8919270833333334 0.2985185185185185 2 0.8911354166666666 0.4561018518518518 2 0.8863802083333333 0.9238518518518518 2 0.7358750000000001 0.3487314814814815 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.8905729166666667 0.36574074074074076 1 2 0.9135416666666667 0.9009259259259259 0.025 0.1037037037037037 2 0.48151041666666666 0.38842592592592595 0.009895833333333333 0.05648148148148148 1 0.25416666666666665 0.42083333333333334 0.014583333333333334 0.05462962962962963 1 0.3807291666666667 0.24537037037037038 0.010416666666666666 0.04814814814814815 1 0.07291666666666667 0.31851851851851853 0.010416666666666666 0.04814814814814815 1 0.63046875 0.1712962962962963 0.008854166666666666 0.037037037037037035 1 0.2185416666666667 0.37972222222222224 0.021458333333333336 0.05388888888888889 3 0.24916666666666665 0.3971296296296296 0.005625000000000001 0.009074074074074075 2 0.49213541666666666 0.1286574074074074 0.009270833333333334 0.03138888888888889 1 0.46328125 0.5268518518518519 0.0234375 0.06296296296296296 1 0.72734375 0.5412037037037037 0.018229166666666668 0.075

    In the first row I have the 0 label for the pitch + x_center y_center w h + 31 x 3 keypoints. In the following lines I have an object for each row with class x_center y_center w h.

    Starting the train I obtain this warning for each image train: WARNING ⚠️ "images\train\imm1: ignoring corrupt image/label: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (12,) + inhomogeneous part.

    I noticed that the number 12 in "the detected shape" corresponds to the number of rows in the .txt file, but I don't understand where I made a mistake. Thanks for your help!

    1. Glenn JocherUltralytics

      @matteobona thanks for sharing your details. The issue arises because pose estimation in YOLO requires consistent keypoint dimensions across all instances. Since only class 0 has keypoints while others don't, the label rows become inconsistent in length. For pose tasks, all instances must follow the same format defined in kpt_shape. You'll need to either:

      1. Add placeholder keypoints (0,0,0) for non-pose classes, or
      2. Separate pose and detection tasks into different datasets

      For mixed tasks, we recommend training separate detection and pose models. The pose estimation documentation explains the required label format where every instance must have keypoints matching the kpt_shape dimensions.

  6. AdiGaikwadNewpageGitHub

    Hello ultralytics team, I want to use this pose estimation for human action detection like push, walking, falling, fighting, sitting, etc. Can you help me to achieve this? How do I need to prepare dataset for this?

    1. Glenn JocherUltralytics

      @AdiGaikwadNewpage to achieve human action detection using pose estimation, prepare your dataset in the Ultralytics YOLO format, ensuring each image has a corresponding text file with object and keypoint details. For more guidance on dataset preparation, refer to the Pose Estimation Datasets Overview.

    2. Glenn JocherUltralytics

      To detect actions like walking or sitting, you'll need to train a model to recognize pose sequences associated with each action. After training, use the model to predict keypoints and analyze their movement patterns over time to classify actions.

  7. LiJiang97GitHub

    My training set label file is

    0 0.47899 0.45495 0.20011 0.29458 0.38743 0.31863 2 0.54499 0.31326 2 0.57668 0.51331 2 0.38480 0.53105 2 0.38988 0.38655 2 0.54744 0.37768 2 0.56915 0.57890 2 0.39076 0.59874 2

    and I set kpt_shape=[8,3] in the yaml file to detect 8 key points.

    Why does it prompt

    "train: WARNING ⚠️ C:\applications_data\pycharm\yolov8\datasets\object_260_dataset\images\train\image_37.jpg: ignoring corrupt image/label: labels require 14 columns each"

    and

    " File "C:\applications\miniconda3\envs\yolov8\Lib\site-packages\ultralytics\data\dataset.py", line 161, in get_labels len_cls, len_boxes, len_segments = (sum(x) for x in zip(*lengths)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 3, got 0)"?

    thank you

    1. Glenn JocherUltralytics(已编辑)

      @LiJiang97 for kpt_shape: [8, 3] each label row needs 5 + 8×3 = 29 values (class, box x y w h, then x y visibility for each of the 8 keypoints), and your row has exactly that. The warning says "labels require 14 columns", which means the dataset YAML being loaded does not have kpt_shape: [8, 3] (14 columns corresponds to kpt_shape: [3, 3]). Check that the data argument points to the YAML you edited and that the setting is kpt_shape: [8, 3]. For more details, please refer to the pose dataset format documentation.

  8. benicio22GitHub

    Hi, I would like to know if the mAP@0.50 for the key point detection using Ultralytics YOLO v8 is the key point the similarity used by the coco dataset. Also, what does this 0.50 mean? It is not the IoU right?

    1. benicio22GitHub

      Do you guys have a function that calculates the metrics per key point, not overall? How can we do that?

    2. Glenn JocherUltralytics

      @benicio22 currently, Ultralytics YOLOv8 does not provide a built-in function to calculate metrics per key point. However, you can achieve this by extracting the keypoint predictions and ground truth from the results and then computing the metrics manually. For more details on the dataset format and keypoint handling, you can refer to our documentation at https://docs.ultralytics.com/datasets/pose/.

  9. PaulCashman73GitHub

    I've used YoloV8 successfully for instance segmentation of community cats. Now I'm trying to use it for pose estimation and getting nowhere. I have a dataset of 1130 images (output from the instance segmentation) with 20 labeled keypoints. I used the following code to train a pre-trained model:

    model = YOLO('yolov8m-pose.pt') # Use a pre-trained model project = '/content/data/results' name = '100-epochs-'

    results = model.train(data='/content/data/data.yaml', epochs=100, project=project, name=name, patience=0, batch=4, imgsz=640)

    My YAML file is pretty vanilla:

    train: /content/train/images val: /content/valid/images test: /content/test/images

    kpt_shape: [20, 3] flip_idx: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]

    nc: 1 names: ['cat']

    roboflow: workspace: feralcatposeestimation project: feral-cat-pose-detection-6 version: 1 license: CC BY 4.0 url: https://universe.roboflow.com/feralcatposeestimation/feral-cat-pose-detection-6/dataset/1

    My results are dismal. Literally, there are no detections found on any of my test images. The precision and recall for the pose is nearly zero. I feel I must be doing something really wrong to achieve these results.

    I looked at one of the label files. Reformatted and with extra blanks to show the keypoints it is as follows:

    0 # One class 0.33046875. 0.715625 # Are these the x-center and y-center, or lower-left x and y? 0.33203125 0.365625 # Width, height 0.7729089743589743 0.2127192307692308 2 # I assume 2 is visible; it's not documented 0.36486538461538454 0.16201346153846155 2 0.6729679487179487 0.3529711538461539 2 0.3790602564102564 0.32252403846153843 2 0.512726923076923 0.4760394230769231 2 0.4950102564102563 0.5223086538461539 2 0.3149961538461538 0.7037836538461539 0 0.4352820512820513 0.7231221153846155 0 0.19383205128205125 0.7414942307692307 0 0.192075641025641 0.8033788461538462 0 0.5024294871794871 0.954883653846154 2 0.2842653846153846 0.7559990384615386 0 0.279875641025641 0.8072471153846154 0 0.3159115384615384 0.9471951923076924 2 0.36943205128205125 0.7559990384615386 0 0.367675641025641 0.8120817307692307 0 0.367675641025641 0.8662307692307692 0 0.46513461538461537 0.7618 0 0.46601282051282045 0.8140153846153847 0 0.46513461538461537 0.8817019230769232 0

    There are 8 visible keypoints (if 2 means visible), which corresponds to the image. But I don't know how to interpret the x,y coordinates in the second line. If they are x-center and y-center of the bounding box, then some of the keypoints are outside the box. If they are the lower left of the bounding box, then other keypoints are outside the bounding box.

    I;m not sure where to begin to unravel what is happening. Any advice would be appreciated.

    1. Glenn JocherUltralytics

      @PaulCashman73 it looks like you're on the right track with your dataset and YAML configuration. The x and y coordinates in the second line of your label file should indeed be the center of the bounding box, normalized between 0 and 1. If keypoints appear outside the bounding box, it might indicate an issue with the normalization or annotation process. Ensure that all keypoints are correctly normalized and fall within the bounding box coordinates. Additionally, verify that your dataset follows the Ultralytics YOLO format precisely. For further details, you can refer to Ultralytics Pose Estimation Datasets. If the issue persists, try updating to the latest version of the package to see if it resolves any underlying bugs.

    2. PaulCashman73GitHub

      Thanks! This dataset was generated by Roboflow, so I will raise an issue in their support forums to suggest that perhaps the data is not being generated properly.

    3. Glenn JocherUltralytics

      Thanks for the update. It's a good idea to check with Roboflow regarding the dataset generation. Meanwhile, ensure that the x and y coordinates in your label files are indeed the center of the bounding box, normalized between 0 and 1. If the keypoints are outside the bounding box, it might indicate an issue with the normalization process. Also, make sure your dataset follows the Ultralytics YOLO format precisely. If the problem persists, try updating to the latest version of the Ultralytics package.

  10. 1-zymGitHub

    Hello, when I trained yolov8-pose, some indicators were not displayed, only various loss were displayed, but these indicators were included in the result.csv file, such as map50, MAP50-90

    1. 1-zymGitHub

      0 0.605961 0.471792 0.0645781 0.0777917 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.527508 0.501177 0.0356094 0.300104 0.534375 0.377083 2 0.5375 0.372917 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.5375 0.464583 2 0.51875 0.43125 2 0.545312 0.514583 1 0.5375 0.5125 2 0.526563 0.579167 2 0 0 0 0.542188 0.639583 2 0 0 0 0 0.415367 0.431646 0.00692187 0.0497917 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 This is the coco key json label that I converted to yolo format

    2. 1-zymGitHub

      This part of the tutorial is a little unclear for me as a novice, can you provide more detailed yolov8-pose tutorial?

    3. Glenn JocherUltralytics(已编辑)

      @1-zym hi there! Thanks for reaching out. I understand that the tutorial might be a bit overwhelming at first. Let's break it down step-by-step to make it clearer for you.

      Training YOLO26-Pose

      To train a YOLO26-Pose model, you'll need to follow these steps:

      1. Prepare Your Dataset: Ensure your dataset is in the Ultralytics YOLO format. Each image should have a corresponding .txt file with annotations. The format is:

        • <class-index> <x> <y> <width> <height> <px1> <py1> <px2> <py2> ... <pxn> <pyn> for 2D keypoints.
        • <class-index> <x> <y> <width> <height> <px1> <py1> <p1-visibility> <px2> <py2> <p2-visibility> ... <pxn> <pyn> <p2-visibility> for 3D keypoints.
      2. Create a YAML Configuration File: This file defines the dataset paths and other configurations. Here’s an example for a dataset with 17 keypoints:

        path: ../datasets/coco8-pose
        train: images/train
        val: images/val
        kpt_shape: [17, 3]  # 17 keypoints, 3 dimensions (x, y, visibility)
        flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
        names:
          0: person
      3. Train the Model: Use the following Python code to train your model:

        from ultralytics import YOLO
        
        # Load a pretrained model
        model = YOLO("yolo26n-pose.pt")
        
        # Train the model
        results = model.train(data="coco8-pose.yaml", epochs=100, imgsz=640)

        Or, you can use the CLI:

        yolo pose train data=coco8-pose.yaml model=yolo26n-pose.pt epochs=100 imgsz=640
      Monitoring Training Metrics

      During training, you might notice that only loss values are displayed in the console. However, metrics like mAP50 and mAP50-95 are recorded in the results.csv file. This is expected behavior. You can visualize these metrics using tools like TensorBoard or by plotting the CSV data.

      Additional Resources

      For more detailed information, you can refer to the Pose Estimation Datasets Overview. This page provides comprehensive details on dataset formats, YAML configuration, and supported datasets.

      If you encounter any issues or need further assistance, please provide a reproducible example of your problem. This helps us diagnose and resolve the issue more effectively. You can find guidance on creating a minimum reproducible example here.

      Feel free to ask if you have more questions! 😊

    4. 1-zymGitHub

      Thank you for your answer, which is 3D keypoint format? COCO isn't just 2D? In addition, I would like to ask, how can I test the coco test set and generate json files, because the coco official website only accepts json format

    5. 1-zymGitHub

      I solved the problem by reusing the coco-pose.yaml file to download the comment file, probably because some missing key boxes were written after I processed them myself.I want to know how to test with coco, I didn't see it in the tutorial

    6. Glenn JocherUltralytics(已编辑)

      Hi @1-zym, glad to hear you resolved the initial issue! For testing with the COCO test set and generating JSON files, you can follow these steps:

      Testing with COCO Test Set
      1. Prepare the COCO Test Set: Ensure you have the COCO test images and the corresponding annotations in the correct format.

      2. Modify the YAML Configuration: Update your dataset YAML file to include the path to the test images. Here’s an example:

        path: ../datasets/coco
        train: images/train
        val: images/val
        test: images/test  # Add this line for test images
        kpt_shape: [17, 3]  # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible)
        flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
        names:
          0: person
      3. Run the Validation: Use the following command to run the validation on the test set:

        yolo pose val model=path/to/best.pt data=coco-pose.yaml split=test save_json=True
      Generating JSON Files for COCO Evaluation

      To generate the COCO-format JSON predictions, set save_json=True when validating on the test split:

      from ultralytics import YOLO
      
      model = YOLO("path/to/best.pt")
      metrics = model.val(data="coco-pose.yaml", split="test", save_json=True)

      This writes a predictions.json file with image IDs, category IDs, boxes, scores and keypoints in COCO format to the validation run directory (for example runs/pose/val), ready for the evaluation server. Note that model.val() returns a metrics object, not a list of results, so it cannot be iterated.

      Additional Resources

      For more detailed information, you can refer to the Pose Estimation Datasets Overview. This page provides comprehensive details on dataset formats, YAML configuration, and supported datasets.

      If you encounter any issues or need further assistance, please provide a reproducible example of your problem. This helps us diagnose and resolve the issue more effectively. You can find guidance on creating a minimum reproducible example here.

      Feel free to ask if you have more questions! 😊

    7. 1-zymGitHub

      I have the following error according to the way you provided. In addition, can you provide the detailed configuration of pose training? For example: epoch, lr, whether pretrain, etc., the yolom-pose results of training 300 epoches by myself have a big gap with the results provided Ultralytics YOLOv8.2.54 🚀 Python-3.8.19 torch-1.12.1+cu116 CUDA:0 (NVIDIA A40, 45449MiB) YOLOv8m-pose summary (fused): 237 layers, 26447596 parameters, 0 gradients, 81.0 GFLOPs val: Scanning /opt/storage/zhaoyiming/datasets/coco-pose/labels/test2017.cache.. WARNING ⚠️ No labels found in /opt/storage/zhaoyiming/datasets/coco-pose/labels/test2017.cache, training may not work correctly. See https://docs.ultralytics.com/datasets for dataset formatting guidance. Class Images Instances Box(P R mAP50 m all 40670 0 0 0 0 0 0 0 0 0 WARNING ⚠️ no labels found in pose set, can not compute metrics without labels Speed: 0.1ms preprocess, 4.1ms inference, 0.0ms loss, 0.5ms postprocess per image Results saved to /opt/storage/zhaoyiming/ultralytics/runs/pose/val14 Traceback (most recent call last): File "/opt/storage/zhaoyiming/ultralytics/ultralytics/utils/test.py", line 13, in for result in results: TypeError: 'PoseMetrics' object is not iterable

      Process finished with exit code 1

    8. Glenn JocherUltralytics(已编辑)

      Hi @1-zym, thanks for reaching out! It looks like you're encountering an issue with missing labels during validation and a TypeError in your script. Here are a few steps to help resolve these issues:

      Missing Labels Warning

      The warning indicates that no labels were found in your test set. Ensure that your dataset is correctly formatted and that the labels are in the expected location. You can refer to the dataset formatting guidance for more details.

      Configuration for Pose Training

      Here's a detailed example configuration for training a YOLO26-Pose model:

      from ultralytics import YOLO
      
      # Load a pretrained model
      model = YOLO("yolo26m-pose.pt")
      
      # Train the model
      results = model.train(
          data="coco-pose.yaml",  # Path to your dataset YAML file
          epochs=300,             # Number of epochs
          imgsz=640,              # Image size
          lr0=0.01,               # Initial learning rate
          pretrained=True         # Use pretrained weights
      )
      Generating JSON Files for COCO Evaluation

      To generate the COCO-format JSON predictions, set save_json=True when validating on the test split:

      from ultralytics import YOLO
      
      model = YOLO("path/to/best.pt")
      metrics = model.val(data="coco-pose.yaml", split="test", save_json=True)

      This writes a predictions.json file with image IDs, category IDs, boxes, scores and keypoints in COCO format to the validation run directory (for example runs/pose/val), ready for the evaluation server. Note that model.val() returns a metrics object, not a list of results, so it cannot be iterated.

      TypeError: 'PoseMetrics' object is not iterable

      model.val() returns a metrics object, not a list of results, so it cannot be iterated; use save_json=True as shown above. Ensure you are using the latest version of the Ultralytics package and that your script aligns with the expected API.

      If the issue persists, please provide a minimum reproducible example to help us diagnose the problem more effectively. You can find guidance on creating a reproducible example here.

      Feel free to ask if you have more questions! 😊

    9. 1-zymGitHub

      path: ../datasets/coco-pose train: train2017.txt val: val2017.txt test: test-dev2017.txt, 通过coco-pose.yaml,我并没有在我的文件夹成功下载 test-dev2017.txt,是官方没有提供?需要我自己生成? 可以提供一下yolov8-pose的详细训练配置?我自己训练300epoch的yolov8m-pose与你们提供的yolov8m-pose的精度,在coco-pose数据集上map50-90差了将近10个点

    10. 1-zymGitHub

      0 0.620039 0.5939 0.172415 0.14608 0.658793 0.552 2 0.664042 0.55 2 0.656168 0.548 2 0.67979 0.554 2 0 0 0 0.67979 0.57 2 0.648294 0.568 2 0.674541 0.588 2 0.606299 0.582 2 0.627297 0.562 2 0.603675 0.564 2 0.627297 0.616 2 0 0 0 0.595801 0.604 2 0 0 0 0.569554 0.644 2 0 0 0 0 0.385525 0.58557 0.14937 0.12586 0.414698 0.54 1 0.419948 0.538 2 0.412073 0.538 1 0.43832 0.54 2 0 0 0 0.435696 0.558 2 0.417323 0.554 2 0.396325 0.574 2 0 0 0 0 0 0 0 0 0 0.414698 0.608 2 0.39895 0.604 2 0.372703 0.592 2 0 0 0 0.346457 0.638 2 0 0 0 0 0.328898 0.70199 0.0313386 0.06714 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.760577 0.69423 0.0285564 0.05486 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.510709 0.69215 0.0187927 0.04682 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.929554 0.67602 0.0388451 0.01844 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 上面的是我自己通过脚本将coco 关键点标注文件转换为yolo 格式 0 0.620039 0.593900 0.172415 0.146080 0.658793 0.552000 2.000000 0.664042 0.550000 2.000000 0.656168 0.548000 2.000000 0.679790 0.554000 2.000000 0.000000 0.000000 0.000000 0.679790 0.570000 2.000000 0.648294 0.568000 2.000000 0.674541 0.588000 2.000000 0.606299 0.582000 2.000000 0.627297 0.562000 2.000000 0.603675 0.564000 2.000000 0.627297 0.616000 2.000000 0.000000 0.000000 0.000000 0.595801 0.604000 2.000000 0.000000 0.000000 0.000000 0.569554 0.644000 2.000000 0.000000 0.000000 0.000000 0 0.385525 0.585570 0.149370 0.125860 0.414698 0.540000 1.000000 0.419948 0.538000 2.000000 0.412073 0.538000 1.000000 0.438320 0.540000 2.000000 0.000000 0.000000 0.000000 0.435696 0.558000 2.000000 0.417323 0.554000 2.000000 0.396325 0.574000 2.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.414698 0.608000 2.000000 0.398950 0.604000 2.000000 0.372703 0.592000 2.000000 0.000000 0.000000 0.000000 0.346457 0.638000 2.000000 0.000000 0.000000 0.000000 这是我通过coco-pose.yaml下载的,用上面的生成的会对训练结果造成影响?

    11. Glenn JocherUltralytics(已编辑)

      Hi @1-zym, thank you for your detailed message. It looks like you're encountering a few issues related to dataset preparation and training configuration. Let's address them step-by-step.

      Missing test-dev2017.txt

      The test-dev2017.txt file is not provided by default. You need to generate this file yourself by listing the paths to your test images. Here’s a quick way to create it:

      find /path/to/test/images -type f -name "*.jpg" > test-dev2017.txt
      Detailed Training Configuration for YOLO26-Pose

      To achieve optimal results, it's crucial to fine-tune your training configuration. Here’s an example configuration for training YOLO26m-Pose:

      from ultralytics import YOLO
      
      # Load a pretrained model
      model = YOLO("yolo26m-pose.pt")
      
      # Train the model
      results = model.train(
          data="coco-pose.yaml",  # Path to your dataset YAML file
          epochs=300,             # Number of epochs
          imgsz=640,              # Image size
          lr0=0.01,               # Initial learning rate
          pretrained=True,        # Use pretrained weights
          batch=16,               # Batch size
          optimizer='Adam',       # Optimizer
          patience=50,            # Early stopping patience
      )
      Generating JSON Files for COCO Evaluation

      To generate the COCO-format JSON predictions, set save_json=True when validating on the test split:

      from ultralytics import YOLO
      
      model = YOLO("path/to/best.pt")
      metrics = model.val(data="coco-pose.yaml", split="test", save_json=True)

      This writes a predictions.json file with image IDs, category IDs, boxes, scores and keypoints in COCO format to the validation run directory (for example runs/pose/val), ready for the evaluation server. Note that model.val() returns a metrics object, not a list of results, so it cannot be iterated.

      Addressing the TypeError

      model.val() returns a metrics object, not a list of results, so it cannot be iterated; use save_json=True as shown above. Ensure you are using the latest version of the Ultralytics package. If the issue persists, please provide a minimum reproducible example to help us diagnose the problem more effectively. You can find guidance on creating a reproducible example here.

      Feel free to ask if you have more questions! 😊

    12. 1-zymGitHub

      现在我遇到一个问题,我用yolov8m-pose使用相同的训练参数、环境,只是GPU不同,训练300个epoch,不使用预训练,其他参数都是默认,我的精度比他的低了10个点,此外我在自己的卡上使用100个epoch训练yolov8n-pose比使用300个epoch 精度也高了7,8个点,你能为我解答这个问题?

    13. 1-zymGitHub

      task: pose mode: train model: ultralytics/cfg/models/v8/yolov8-pose.yaml data: ultralytics/cfg/datasets/coco-pose.yaml epochs: 300 time: null patience: 100 batch: 64 imgsz: 800 save: true save_period: -1 cache: true device: '1' workers: 8 project: null name: train35 exist_ok: false pretrained: false optimizer: SGD verbose: true seed: 0 deterministic: true single_cls: false rect: false cos_lr: false close_mosaic: 10 resume: false amp: true fraction: 1.0 profile: false freeze: null multi_scale: false overlap_mask: true mask_ratio: 4 dropout: 0.0 val: true split: val save_json: false save_hybrid: false conf: null iou: 0.7 max_det: 300 half: false dnn: false plots: true source: null vid_stride: 1 stream_buffer: false visualize: false augment: false agnostic_nms: false classes: null retina_masks: false embed: null show: false save_frames: false save_txt: false save_conf: false save_crop: false show_labels: true show_conf: true show_boxes: true line_width: null format: torchscript keras: false optimize: false int8: false dynamic: false simplify: false opset: null workspace: 4 nms: false lr0: 0.01 lrf: 0.01 momentum: 0.937 weight_decay: 0.0005 warmup_epochs: 3.0 warmup_momentum: 0.8 warmup_bias_lr: 0.1 box: 7.5 cls: 0.5 dfl: 1.5 pose: 12.0 kobj: 1.0 label_smoothing: 0.0 nbs: 64 hsv_h: 0.015 hsv_s: 0.7 hsv_v: 0.4 degrees: 0.0 translate: 0.1 scale: 0.5 shear: 0.0 perspective: 0.0 flipud: 0.0 fliplr: 0.5 bgr: 0.0 mosaic: 1.0 mixup: 0.0 copy_paste: 0.0 auto_augment: randaugment erasing: 0.4 crop_fraction: 1.0 cfg: null tracker: botsort.yaml save_dir: /opt/storage/zhaoyiming/ultralytics/runs/pose/train35 这是参数文件

    14. Glenn JocherUltralytics

      It sounds like you're experiencing significant variations in performance despite using similar training parameters. Differences in GPU architecture can sometimes lead to variations in training results due to factors like hardware-specific optimizations and floating-point precision. Additionally, training stability and convergence can be influenced by random initialization and data augmentation. To address this, ensure you're using the latest version of the Ultralytics package and try setting a fixed random seed for reproducibility. If the issue persists, consider experimenting with different learning rates, batch sizes, and optimizers to find the optimal configuration for your specific setup.

    15. 1-zymGitHub

      为什么yolov8n-pose训练100个epoch会比300个epoch要好呢?lr0=0.01 lrf=0.01 ,其他的超参数都一样,都是默认,我不是很明白它的学习率是怎么衰减的。

    16. Glenn JocherUltralytics

      The difference in performance between training for 100 epochs and 300 epochs could be due to overfitting. With a learning rate of 0.01 and default parameters, the model might be learning too quickly and overfitting to the training data. To address this, consider using a learning rate scheduler or early stopping. Additionally, ensure that your dataset is well-augmented and that the training process is consistent across different GPUs. For more details on learning rate schedules and training strategies, you can refer to the Ultralytics documentation.