포즈 추정 데이터셋 개요
지원되는 데이터셋 형식#
Ultralytics YOLO 형식#
YOLO 포즈 모델 학습에 사용되는 데이터셋 레이블 형식은 다음과 같습니다.
- 이미지마다 텍스트 파일 하나: 데이터셋의 각 이미지에는 이미지 파일과 이름이 같고 확장자가 ".txt"인 텍스트 파일이 하나씩 대응됩니다.
- 객체마다 한 행: 텍스트 파일의 각 행은 이미지의 객체 인스턴스 하나에 해당합니다.
- 행마다 객체 정보 포함: 각 행에는 객체 인스턴스에 관한 다음 정보가 포함됩니다:
- 객체 클래스 인덱스: 객체의 클래스를 나타내는 정수입니다(예: 사람은 0, 자동차는 1 등).
- 객체 중심 좌표: 객체 중심의 x 및 y 좌표이며, 0~1 범위로 정규화됩니다.
- 객체 너비와 높이: 객체의 너비와 높이이며, 0~1 범위로 정규화됩니다.
- 객체 키포인트 좌표: 객체의 키포인트는 0과 1 사이의 값으로 정규화됩니다.
다음은 포즈 추정 작업의 라벨 형식 예시입니다:
2D 키포인트 형식
<class-index> <x> <y> <width> <height> <px1> <py1> <px2> <py2> ... <pxn> <pyn>키포인트 가시성 형식(각 포인트의 가시성 포함)
<class-index> <x> <y> <width> <height> <px1> <py1> <p1-visibility> <px2> <py2> <p2-visibility> ... <pxn> <pyn> <pn-visibility>이 형식에서 <class-index>은 객체의 클래스 인덱스이고, <x> <y> <width> <height>은 경계 상자의 정규화된 좌표이며, <px1> <py1> <px2> <py2> ... <pxn> <pyn>은 키포인트의 정규화된 좌표입니다. 가시성 채널은 선택 사항이지만 가림을 주석 처리하는 데이터셋에 유용합니다.
데이터셋 YAML 형식#
Ultralytics 프레임워크는 포즈 추정 모델 학습을 위한 데이터셋 및 모델 구성을 정의할 때 YAML 파일 형식을 사용합니다. 다음은 포즈 데이터셋을 정의하는 데 사용되는 YAML 형식의 예시입니다:
# Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license
# COCO8-pose dataset (first 8 images from COCO train2017) by Ultralytics
# Documentation: https://docs.ultralytics.com/datasets/pose/coco8-pose
# Example usage: yolo train data=coco8-pose.yaml
# parent
# ├── ultralytics
# └── datasets
# └── coco8-pose ← downloads here (1 MB)
# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: coco8-pose # dataset root dir
train: images/train # train images (relative to 'path') 4 images
val: images/val # val images (relative to 'path') 4 images
test: # test images (optional)
# Keypoints
kpt_shape: [17, 3] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible)
flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
# Classes
names:
0: person
# Keypoint names per class
kpt_names:
0:
- nose
- left_eye
- right_eye
- left_ear
- right_ear
- left_shoulder
- right_shoulder
- left_elbow
- right_elbow
- left_wrist
- right_wrist
- left_hip
- right_hip
- left_knee
- right_knee
- left_ankle
- right_ankle
# Download script/URL (optional)
download: https://github.com/ultralytics/assets/releases/download/v0.0.0/coco8-pose.ziptrain, val, test 필드는 각각 학습, 검증, 테스트 이미지를 가리킵니다. 각 필드는 디렉터리, 이미지 파일, 디렉터리 또는 이미지 파일 목록, 또는 줄마다 이미지 경로 하나가 지정된 UTF-8 *.txt 파일을 입력으로 받습니다(경로가 ./로 시작하면 *.txt 파일을 기준으로 확인합니다). *.txt 파일은 디렉터리의 일부만 선택해 학습하거나, 라벨이 없는 이미지를 건너뛰거나, 여러 소스의 이미지를 하나의 split으로 결합할 때 유용합니다.
path: datasets/coco8-pose # dataset root
train: train.txt # a directory, an image, a list e.g. [images/a, images/b], or a *.txt file
val: val.txt
kpt_shape: [17, 3] # required for pose datasets
names:
0: personnames은 클래스 이름으로 구성된 딕셔너리입니다. 이름의 순서는 YOLO 데이터셋 파일에 지정된 객체 클래스 인덱스의 순서와 일치해야 합니다.
(선택 사항) flip_idx은 각 키포인트를 거울상에 매핑하므로, 수평 뒤집기 증강을 적용해도 사람의 몸이나 얼굴처럼 대칭인 스켈레톤에서 좌우가 일관되게 유지됩니다. [왼쪽 눈, 오른쪽 눈, 코, 왼쪽 입, 오른쪽 입] = [0, 1, 2, 3, 4]로 인덱싱된 다섯 개의 얼굴 랜드마크에서는 flip_idx이 [1, 0, 2, 4, 3]입니다. 즉, 좌우 쌍인 0-1과 3-4는 서로 바뀌고 코는 자체 인덱스를 유지합니다.
(선택 사항) kpt_oks_sigmas은 학습 및 검증 중에 사용되는 사용자 지정 키포인트별 OKS 시그마를 설정합니다(예: [0.26, 0.25, 0.25, ...]). 목록 길이는 kpt_shape의 키포인트 수인 N과 같아야 하며, 모든 값은 양수여야 합니다. 이 값을 지정하지 않으면 kpt_shape: [17, 3]에는 COCO 17개 키포인트 시그마가 사용되고, 그 외에는 균일한 1/N이 사용됩니다.
사용법#
from ultralytics import YOLO
# 모델 로드
model = YOLO("yolo26n-pose.pt") # 사전 학습된 모델을 불러옵니다(학습에 권장).
# 모델 학습
results = model.train(data="coco8-pose.yaml", epochs=100, imgsz=640)지원되는 데이터셋#
이 섹션에서는 Ultralytics YOLO 형식과 호환되며 포즈 추정 모델 학습에 사용할 수 있는 데이터셋을 소개합니다. 대부분의 데이터셋은 Ultralytics Platform에서도 호스팅되므로 이미지와 주석을 살펴보고, 데이터셋 통계를 확인하고, 클라우드 학습을 위해 복제할 수 있습니다.
COCO-Pose#
- 설명: COCO-Pose는 키포인트 주석이 있는 사람이 포함된 COCO 2017 이미지를 다루는 대규모 인간 포즈 추정 데이터셋입니다.
- 라벨 형식: 위에서 설명한 Ultralytics YOLO 형식과 동일하며, 인간 포즈 키포인트가 포함됩니다.
- 클래스 수: 1개(사람).
- 키포인트: 코, 눈, 귀, 어깨, 팔꿈치, 손목, 엉덩이, 무릎, 발목을 포함한 17가지 키포인트 유형이며, 각 키포인트에는 가시성 차원이 포함됩니다.
- 용도: 인간 포즈 추정 모델 학습에 적합합니다.
- 추가 참고 사항: 이 데이터셋은 COCO Keypoints 2017 챌린지를 기반으로 하며, 58,945개 이미지에 156,165명의 주석이 포함되어 있습니다.
- COCO-Pose 자세히 알아보기
COCO8-Pose#
- 설명: Ultralytics COCO8-Pose는 COCO train 2017 세트의 첫 8개 이미지로 구성된 소규모 포즈 추정 데이터셋으로, 학습용 4개와 검증용 4개 이미지가 포함됩니다.
- 라벨 형식: 위에서 설명한 Ultralytics YOLO 형식과 동일하며, 인간 포즈 키포인트가 포함됩니다.
- 클래스 수: 1개(사람).
- 키포인트: 코, 눈, 귀, 어깨, 팔꿈치, 손목, 엉덩이, 무릎, 발목을 포함한 17가지 키포인트 유형이며, 각 키포인트에는 가시성 차원이 포함됩니다.
- 용도: 포즈 추정 모델을 테스트하고 디버깅하거나 새로운 키포인트 감지 접근 방식을 실험하는 데 적합합니다.
- 추가 참고 사항: COCO8-Pose는 기본 동작 확인 및 CI 검사에 이상적입니다.
- COCO8-Pose 자세히 알아보기
Dog-Pose#
- 설명: Ultralytics Dog-Pose 데이터셋에는 개의 키포인트 추정을 위한 학습 이미지 6,773개와 검증 이미지 1,703개가 포함되어 있습니다.
- 라벨 형식: Ultralytics YOLO 형식을 따르며, 개의 해부학적 특징에 특화된 여러 키포인트의 주석이 포함됩니다.
- 클래스 수: 1개(개).
- 키포인트: 각 키포인트에 가시성 차원이 포함된 24개 키포인트로, 사지, 관절, 머리 위치 등 개의 포즈에 맞게 구성되어 있습니다.
- 용도: 연구부터 실제 애플리케이션까지 다양한 시나리오에서 개의 포즈를 추정하는 모델 학습에 적합합니다.
- 추가 참고 사항: 원본 이미지는 Stanford Dogs Dataset에서 가져왔습니다.
- Dog-Pose 자세히 알아보기
손 키포인트#
- 설명: Ultralytics 손 키포인트 데이터셋은 총 26,768개 이미지로 구성되며, 학습용 18,776개와 검증용 7,992개로 나뉩니다.
- 라벨 형식: 위에서 설명한 Ultralytics YOLO 형식과 동일하지만, 사람 손의 21개 키포인트와 가시성 차원이 포함됩니다.
- 클래스 수: 1개(손).
- 키포인트: 21개 키포인트.
- 용도: 사람 손 포즈 추정 및 제스처 인식에 적합합니다.
- 추가 참고 사항: 일관된 라벨링을 위해 Google MediaPipe를 사용해 키포인트 주석을 생성합니다.
- 손 키포인트 자세히 알아보기
Tiger-Pose#
- 설명: Ultralytics Tiger-Pose 데이터셋은 YouTube 동영상에서 가져온 263개 이미지로 구성되며, 학습용 210개와 검증용 53개로 나뉩니다.
- 라벨 형식: 위에서 설명한 Ultralytics YOLO 형식과 동일하며, 동물 포즈를 위한 키포인트 12개가 포함되고 가시성 차원은 없습니다.
- 클래스 수: 1개(호랑이).
- 키포인트: 12개 키포인트.
- 용도: 동물 포즈 또는 사람을 대상으로 하지 않는 다른 포즈 작업에 적합합니다.
- 추가 참고 사항: AGPL-3.0 라이선스에 따라 배포됩니다.
- Tiger-Pose 자세히 알아보기
사용자 데이터셋 추가#
자체 데이터셋을 보유하고 있으며 Ultralytics YOLO 형식으로 포즈 추정 모델을 학습하는 데 사용하려면, 위의 "Ultralytics YOLO 형식"에 명시된 형식을 따르는지 확인하세요. 주석을 필수 형식으로 변환하고 YAML 구성 파일에 경로, 클래스 수, 클래스 이름을 지정하세요.
변환 단계를 완전히 건너뛰려면 Ultralytics Platform에 원본 이미지를 업로드하고, 브라우저에서 키포인트에 주석을 추가한 다음, 결과 데이터셋으로 바로 학습할 수 있습니다.
변환 도구#
Ultralytics는 널리 사용되는 COCO 데이터셋 형식의 라벨을 YOLO 형식으로 변환하는 도구를 제공합니다:
from ultralytics.data.converter import convert_coco
convert_coco(labels_dir="path/to/coco/annotations/", use_keypoints=True)이 변환 도구를 사용하면 COCO 데이터셋 또는 COCO 형식의 모든 데이터셋을 Ultralytics YOLO 형식으로 변환할 수 있습니다. use_keypoints 매개변수는 변환된 라벨에 키포인트(포즈 추정용)를 포함할지 여부를 지정합니다.
자주 묻는 질문#
포즈 추정 데이터셋을 위한 Ultralytics YOLO 형식에서는 각 이미지에 해당하는 텍스트 파일로 라벨을 지정합니다. 텍스트 파일의 각 행에는 객체 인스턴스에 대한 정보가 저장됩니다:
- 객체 클래스 인덱스
- 객체 중심 좌표(정규화된 x 및 y)
- 객체 너비와 높이(정규화됨)
- 객체 키포인트 좌표(정규화된 pxn 및 pyn)
2D 포즈의 키포인트에는 정규화된 x 및 y 좌표가 포함됩니다. 가시성 차원이 있는 경우 각 키포인트에는 가시성 플래그도 포함됩니다. 자세한 내용은 Ultralytics YOLO 형식을 참조하세요.
데이터셋을 추가하려면:
-
주석을 Ultralytics YOLO 형식으로 변환합니다.
-
데이터셋 경로, 클래스 수, 클래스 이름을 지정하는 YAML 구성 파일을 만듭니다.
-
구성 파일을 사용해 모델을 학습합니다:
from ultralytics import YOLO model = YOLO("yolo26n-pose.pt") results = model.train(data="your-dataset.yaml", epochs=100, imgsz=640)전체 단계는 자체 데이터셋 추가 섹션을 참조하세요.
-
Ultralytics YOLO의 데이터셋 YAML 파일은 학습을 위한 데이터셋 및 모델 구성을 정의합니다. 학습, 검증, 테스트 이미지의 경로, 키포인트 형태, 클래스 이름 및 기타 구성 옵션을 지정합니다. 이러한 구조화된 형식은 데이터셋 관리와 모델 학습을 간소화합니다. 다음은 YAML 형식의 예시입니다:
# Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license # COCO8-pose dataset (first 8 images from COCO train2017) by Ultralytics # Documentation: https://docs.ultralytics.com/datasets/pose/coco8-pose # Example usage: yolo train data=coco8-pose.yaml # parent # ├── ultralytics # └── datasets # └── coco8-pose ← downloads here (1 MB) # Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..] path: coco8-pose # dataset root dir train: images/train # train images (relative to 'path') 4 images val: images/val # val images (relative to 'path') 4 images test: # test images (optional) # Keypoints kpt_shape: [17, 3] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible) flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15] # Classes names: 0: person # Keypoint names per class kpt_names: 0: - nose - left_eye - right_eye - left_ear - right_ear - left_shoulder - right_shoulder - left_elbow - right_elbow - left_wrist - right_wrist - left_hip - right_hip - left_knee - right_knee - left_ankle - right_ankle # Download script/URL (optional) download: https://github.com/ultralytics/assets/releases/download/v0.0.0/coco8-pose.zipYAML 구성 파일 작성에 대한 자세한 내용은 데이터셋 YAML 형식을 참조하세요.
Ultralytics는 키포인트 정보를 포함해 COCO 데이터셋 라벨을 YOLO 형식으로 변환하는 도구를 제공합니다:
from ultralytics.data.converter import convert_coco convert_coco(labels_dir="path/to/coco/annotations/", use_keypoints=True)이 도구를 사용하면 COCO 형식 데이터셋을 YOLO 프로젝트로 가져올 수 있습니다.
댓글69
질문을 하거나 결과를 공유하거나 다른 사람을 도와주세요.
OfekDayan
I trained pose models (n - x) on a custom data - pitch on a football playground (skeleton with 32 points). Most of the key-points are detected on the correct location, but they have wrong ids.
What could be the reason?
UltralyticsAssistantUltralytics
👋 Hello, thank you for your interest in Ultralytics 🚀! We recommend checking out the Docs for detailed information about pose estimation datasets. Specifically, you can explore the Pose Datasets section, where you'll find guidance on supported formats like COCO-Pose, COCO8-Pose, Tiger-Pose, and instructions for adding custom datasets.
If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug the issue.
For custom training ❓ Questions, please include as much detail as possible, such as dataset examples, training logs, or specific challenges. Ensure you're following our Tips for Best Training Results for optimal performance.
Join the Ultralytics community where it suits you best:
- For real-time discussions, visit Discord 🎧.
- Explore in-depth topics on Discourse.
- Share insights and connect on our Subreddit.
Upgrade
To ensure you are using the latest features and fixes, upgrade to the latest
ultralyticspackage with:pip install -U ultralyticsVerify your environment includes all requirements and is running Python>=3.8 and PyTorch>=1.8.
Environments
Run YOLO in any of these up-to-date verified environments with preinstalled dependencies:
- Notebooks with free GPUs:
- Google Cloud Deep Learning VM: See GCP Quickstart Guide.
- Amazon Deep Learning AMI: See AWS Quickstart Guide.
- Docker Image: See Docker Quickstart Guide
.
Status
If this badge is green, all Ultralytics CI tests are passing. CI tests validate the functionality of YOLO Modes and Tasks across macOS, Windows, and Ubuntu.
This is an automated response, but an Ultralytics engineer will assist you shortly. Let us know if you need further guidance! 😊
Glenn JocherUltralytics@Alebep Currently in the Ultralytics YOLO framework, all classes within a single dataset must use the same number of keypoints as defined by the
kpt_shapeparameter in your YAML file. This is a framework-level constraint where the model architecture expects a consistent keypoint structure across all detected objects.If you need to work with objects requiring different numbers of keypoints (like your dog with 5 and turtle with 3 example), you would need to:
- Train separate models for each keypoint configuration using different datasets
- Or use a unified keypoint count approach where you define all possible keypoints and mark unused ones as invisible for certain classes
This is because the pose estimation head in YOLO is structured to predict a fixed number of keypoints for all objects. Each dataset's configuration defines this through the
kpt_shapeparameter, which applies uniformly to all classes in that dataset.For examples of different keypoint configurations, you can look at our existing pose datasets like Dog-Pose (24 keypoints) and Tiger-Pose (12 keypoints), each optimized for their specific subject.
Alebep
Can the number of keypoints vary by class? For example, could the 'dog' class have 5 keypoints while the 'turtle' class has only 3? Or do all classes need to have the same number of keypoints?
Glenn JocherUltralyticsHi there! 👋 In Ultralytics YOLO pose estimation, all classes in a single dataset must share the same number of keypoints as defined by the
kpt_shapeparameter in the dataset YAML file. This configuration applies uniformly across all classes within that dataset. For mixed keypoint requirements across object types, you'll need to train separate models on different datasets. Check out our Pose Estimation Datasets Guide for details on configuring keypoint dimensions.1
foolishcqx
首先,非常感谢你们的工作!最近我在进行自定义的关键点检测时遇到了一个问题,我们的模型在训练集和验证集上的效果在训练过程中指标都表现的很好,但是在我测试效果很糟糕。是否有什么技巧可以改善这个问题? 数据标签格式: 0 0.528905 0.425485 0.501863 0.742139 0.385417 0.384615 0.406250 0.377622 0.429688 0.374126 0.450521 0.384615 0.427083 0.391608 0.403646 0.395105 0.565104 0.388112 0.585938 0.377622 0.609375 0.381119 0.630208 0.391608 0.609375 0.398601 0.585938 0.395105 0.450521 0.636364 0.486979 0.632867 0.510417 0.636364 0.531250 0.632867 0.565104 0.632867 0.531250 0.632867 0.510417 0.636364 0.486979 0.632867 配置文件的格式:
path: /data/zhongjiafeng/ultralytics-main/posedataset # dataset root dir train: /data/zhongjiafeng/ultralytics-main/posedataset/train/images # train images (relative to 'path') 4 images val: /data/zhongjiafeng/ultralytics-main/posedataset/val/images # val images (relative to 'path') 4 images test: # test images (optional)
Keypoints
kpt_shape: [20, 2] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible)
Classes
names: 0: face
我们是在加载yolo11s-pose.pt模型的基础上进行训练的
Glenn JocherUltralytics@foolishcqx 感谢您的详细描述!从您的配置来看,关键点维度设置为2但缺少必要的flip_idx对称翻转索引,这会影响数据增强效果。建议参考COCO-Pose的YAML配置,为面部对称关键点(如双眼/嘴角)添加正确的flip_idx,并验证关键点坐标是否已归一化到0-1范围。具体可查看Pose数据集格式文档中的对称点配置说明。
matteobona
Hi, I was trying to train a custom pose model on a dataset with 4 classes. The first class has 31 keypoints, while the others do not have keypoints. I created my YAML file as follows:
train: "images\train" # train images val: "images\valid" #val images kpt_shape: [31, 3] nc: 4 names: ['pitch', 'player', 'referee', 'ball']
And this is an example of a .txt file of annotations: 0 0.5002604166666667 0.5421296296296296 0.9994791666666667 0.8953703703703704 0.367078125 0.12500925925925926 2 0.2983958333333333 0.1687037037037037 2 0.22916145833333335 0.2129722222222222 2 0.0834375 0.3101851851851852 2 0 0 0 0 0 0 0.283984375 0.22016666666666668 2 0.14989583333333334 0.32462962962962966 2 0.463375 0.18461111111111111 2 0.39940625 0.2422314814814815 2 0.30180208333333336 0.3329259259259259 2 0.1875 0.4439814814814815 2 0.89265625 0.18564814814814815 2 0.8919270833333334 0.2985185185185185 2 0.8911354166666666 0.4561018518518518 2 0.8863802083333333 0.9238518518518518 2 0.7358750000000001 0.3487314814814815 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.8905729166666667 0.36574074074074076 1 2 0.9135416666666667 0.9009259259259259 0.025 0.1037037037037037 2 0.48151041666666666 0.38842592592592595 0.009895833333333333 0.05648148148148148 1 0.25416666666666665 0.42083333333333334 0.014583333333333334 0.05462962962962963 1 0.3807291666666667 0.24537037037037038 0.010416666666666666 0.04814814814814815 1 0.07291666666666667 0.31851851851851853 0.010416666666666666 0.04814814814814815 1 0.63046875 0.1712962962962963 0.008854166666666666 0.037037037037037035 1 0.2185416666666667 0.37972222222222224 0.021458333333333336 0.05388888888888889 3 0.24916666666666665 0.3971296296296296 0.005625000000000001 0.009074074074074075 2 0.49213541666666666 0.1286574074074074 0.009270833333333334 0.03138888888888889 1 0.46328125 0.5268518518518519 0.0234375 0.06296296296296296 1 0.72734375 0.5412037037037037 0.018229166666666668 0.075
In the first row I have the 0 label for the pitch + x_center y_center w h + 31 x 3 keypoints. In the following lines I have an object for each row with class x_center y_center w h.
Starting the train I obtain this warning for each image train: WARNING ⚠️ "images\train\imm1: ignoring corrupt image/label: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (12,) + inhomogeneous part.
I noticed that the number 12 in "the detected shape" corresponds to the number of rows in the .txt file, but I don't understand where I made a mistake. Thanks for your help!
Glenn JocherUltralytics@matteobona thanks for sharing your details. The issue arises because pose estimation in YOLO requires consistent keypoint dimensions across all instances. Since only class 0 has keypoints while others don't, the label rows become inconsistent in length. For pose tasks, all instances must follow the same format defined in kpt_shape. You'll need to either:
- Add placeholder keypoints (0,0,0) for non-pose classes, or
- Separate pose and detection tasks into different datasets
For mixed tasks, we recommend training separate detection and pose models. The pose estimation documentation explains the required label format where every instance must have keypoints matching the kpt_shape dimensions.
AdiGaikwadNewpage
Hello ultralytics team, I want to use this pose estimation for human action detection like push, walking, falling, fighting, sitting, etc. Can you help me to achieve this? How do I need to prepare dataset for this?
Glenn JocherUltralytics@AdiGaikwadNewpage to achieve human action detection using pose estimation, prepare your dataset in the Ultralytics YOLO format, ensuring each image has a corresponding text file with object and keypoint details. For more guidance on dataset preparation, refer to the Pose Estimation Datasets Overview.
AdiGaikwadNewpage
Hello @glenn-jocher, Thank you for your help in data preparation. Can you tell me now that what I need to do to detect action after training a model?
Glenn JocherUltralyticsTo detect actions like walking or sitting, you'll need to train a model to recognize pose sequences associated with each action. After training, use the model to predict keypoints and analyze their movement patterns over time to classify actions.
LiJiang97
My training set label file is
0 0.47899 0.45495 0.20011 0.29458 0.38743 0.31863 2 0.54499 0.31326 2 0.57668 0.51331 2 0.38480 0.53105 2 0.38988 0.38655 2 0.54744 0.37768 2 0.56915 0.57890 2 0.39076 0.59874 2
and I set kpt_shape=[8,3] in the yaml file to detect 8 key points.
Why does it prompt
"train: WARNING ⚠️ C:\applications_data\pycharm\yolov8\datasets\object_260_dataset\images\train\image_37.jpg: ignoring corrupt image/label: labels require 14 columns each"
and
" File "C:\applications\miniconda3\envs\yolov8\Lib\site-packages\ultralytics\data\dataset.py", line 161, in get_labels len_cls, len_boxes, len_segments = (sum(x) for x in zip(*lengths)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: not enough values to unpack (expected 3, got 0)"?
thank you
Glenn JocherUltralytics(수정됨)@LiJiang97 for
kpt_shape: [8, 3]each label row needs 5 + 8×3 = 29 values (class, box x y w h, then x y visibility for each of the 8 keypoints), and your row has exactly that. The warning says "labels require 14 columns", which means the dataset YAML being loaded does not havekpt_shape: [8, 3](14 columns corresponds tokpt_shape: [3, 3]). Check that thedataargument points to the YAML you edited and that the setting iskpt_shape: [8, 3]. For more details, please refer to the pose dataset format documentation.
benicio22
Hi, I would like to know if the mAP@0.50 for the key point detection using Ultralytics YOLO v8 is the key point the similarity used by the coco dataset. Also, what does this 0.50 mean? It is not the IoU right?
benicio22
Do you guys have a function that calculates the metrics per key point, not overall? How can we do that?
Glenn JocherUltralytics@benicio22 currently, Ultralytics YOLOv8 does not provide a built-in function to calculate metrics per key point. However, you can achieve this by extracting the keypoint predictions and ground truth from the results and then computing the metrics manually. For more details on the dataset format and keypoint handling, you can refer to our documentation at https://docs.ultralytics.com/datasets/pose/.
PaulCashman73
I've used YoloV8 successfully for instance segmentation of community cats. Now I'm trying to use it for pose estimation and getting nowhere. I have a dataset of 1130 images (output from the instance segmentation) with 20 labeled keypoints. I used the following code to train a pre-trained model:
model = YOLO('yolov8m-pose.pt') # Use a pre-trained model project = '/content/data/results' name = '100-epochs-'
results = model.train(data='/content/data/data.yaml', epochs=100, project=project, name=name, patience=0, batch=4, imgsz=640)
My YAML file is pretty vanilla:
train: /content/train/images val: /content/valid/images test: /content/test/images
kpt_shape: [20, 3] flip_idx: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
nc: 1 names: ['cat']
roboflow: workspace: feralcatposeestimation project: feral-cat-pose-detection-6 version: 1 license: CC BY 4.0 url: https://universe.roboflow.com/feralcatposeestimation/feral-cat-pose-detection-6/dataset/1
My results are dismal. Literally, there are no detections found on any of my test images. The precision and recall for the pose is nearly zero. I feel I must be doing something really wrong to achieve these results.
I looked at one of the label files. Reformatted and with extra blanks to show the keypoints it is as follows:
0 # One class 0.33046875. 0.715625 # Are these the x-center and y-center, or lower-left x and y? 0.33203125 0.365625 # Width, height 0.7729089743589743 0.2127192307692308 2 # I assume 2 is visible; it's not documented 0.36486538461538454 0.16201346153846155 2 0.6729679487179487 0.3529711538461539 2 0.3790602564102564 0.32252403846153843 2 0.512726923076923 0.4760394230769231 2 0.4950102564102563 0.5223086538461539 2 0.3149961538461538 0.7037836538461539 0 0.4352820512820513 0.7231221153846155 0 0.19383205128205125 0.7414942307692307 0 0.192075641025641 0.8033788461538462 0 0.5024294871794871 0.954883653846154 2 0.2842653846153846 0.7559990384615386 0 0.279875641025641 0.8072471153846154 0 0.3159115384615384 0.9471951923076924 2 0.36943205128205125 0.7559990384615386 0 0.367675641025641 0.8120817307692307 0 0.367675641025641 0.8662307692307692 0 0.46513461538461537 0.7618 0 0.46601282051282045 0.8140153846153847 0 0.46513461538461537 0.8817019230769232 0
There are 8 visible keypoints (if 2 means visible), which corresponds to the image. But I don't know how to interpret the x,y coordinates in the second line. If they are x-center and y-center of the bounding box, then some of the keypoints are outside the box. If they are the lower left of the bounding box, then other keypoints are outside the bounding box.
I;m not sure where to begin to unravel what is happening. Any advice would be appreciated.
Glenn JocherUltralytics@PaulCashman73 it looks like you're on the right track with your dataset and YAML configuration. The x and y coordinates in the second line of your label file should indeed be the center of the bounding box, normalized between 0 and 1. If keypoints appear outside the bounding box, it might indicate an issue with the normalization or annotation process. Ensure that all keypoints are correctly normalized and fall within the bounding box coordinates. Additionally, verify that your dataset follows the Ultralytics YOLO format precisely. For further details, you can refer to Ultralytics Pose Estimation Datasets. If the issue persists, try updating to the latest version of the package to see if it resolves any underlying bugs.
PaulCashman73
Thanks! This dataset was generated by Roboflow, so I will raise an issue in their support forums to suggest that perhaps the data is not being generated properly.
Glenn JocherUltralyticsThanks for the update. It's a good idea to check with Roboflow regarding the dataset generation. Meanwhile, ensure that the x and y coordinates in your label files are indeed the center of the bounding box, normalized between 0 and 1. If the keypoints are outside the bounding box, it might indicate an issue with the normalization process. Also, make sure your dataset follows the Ultralytics YOLO format precisely. If the problem persists, try updating to the latest version of the Ultralytics package.
1-zym
Hello, when I trained yolov8-pose, some indicators were not displayed, only various loss were displayed, but these indicators were included in the result.csv file, such as map50, MAP50-90
1-zym
0 0.605961 0.471792 0.0645781 0.0777917 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.527508 0.501177 0.0356094 0.300104 0.534375 0.377083 2 0.5375 0.372917 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.5375 0.464583 2 0.51875 0.43125 2 0.545312 0.514583 1 0.5375 0.5125 2 0.526563 0.579167 2 0 0 0 0.542188 0.639583 2 0 0 0 0 0.415367 0.431646 0.00692187 0.0497917 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 This is the coco key json label that I converted to yolo format
1-zym
This part of the tutorial is a little unclear for me as a novice, can you provide more detailed yolov8-pose tutorial?
Glenn JocherUltralytics(수정됨)@1-zym hi there! Thanks for reaching out. I understand that the tutorial might be a bit overwhelming at first. Let's break it down step-by-step to make it clearer for you.
Training YOLO26-Pose
To train a YOLO26-Pose model, you'll need to follow these steps:
-
Prepare Your Dataset: Ensure your dataset is in the Ultralytics YOLO format. Each image should have a corresponding
.txtfile with annotations. The format is:<class-index> <x> <y> <width> <height> <px1> <py1> <px2> <py2> ... <pxn> <pyn>for 2D keypoints.<class-index> <x> <y> <width> <height> <px1> <py1> <p1-visibility> <px2> <py2> <p2-visibility> ... <pxn> <pyn> <p2-visibility>for 3D keypoints.
-
Create a YAML Configuration File: This file defines the dataset paths and other configurations. Here’s an example for a dataset with 17 keypoints:
path: ../datasets/coco8-pose train: images/train val: images/val kpt_shape: [17, 3] # 17 keypoints, 3 dimensions (x, y, visibility) flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15] names: 0: person -
Train the Model: Use the following Python code to train your model:
from ultralytics import YOLO # Load a pretrained model model = YOLO("yolo26n-pose.pt") # Train the model results = model.train(data="coco8-pose.yaml", epochs=100, imgsz=640)Or, you can use the CLI:
yolo pose train data=coco8-pose.yaml model=yolo26n-pose.pt epochs=100 imgsz=640
Monitoring Training Metrics
During training, you might notice that only loss values are displayed in the console. However, metrics like mAP50 and mAP50-95 are recorded in the
results.csvfile. This is expected behavior. You can visualize these metrics using tools like TensorBoard or by plotting the CSV data.Additional Resources
For more detailed information, you can refer to the Pose Estimation Datasets Overview. This page provides comprehensive details on dataset formats, YAML configuration, and supported datasets.
If you encounter any issues or need further assistance, please provide a reproducible example of your problem. This helps us diagnose and resolve the issue more effectively. You can find guidance on creating a minimum reproducible example here.
Feel free to ask if you have more questions! 😊
-
1-zym
Thank you for your answer, which is 3D keypoint format? COCO isn't just 2D? In addition, I would like to ask, how can I test the coco test set and generate json files, because the coco official website only accepts json format
1-zym
I solved the problem by reusing the coco-pose.yaml file to download the comment file, probably because some missing key boxes were written after I processed them myself.I want to know how to test with coco, I didn't see it in the tutorial
Glenn JocherUltralytics(수정됨)Hi @1-zym, glad to hear you resolved the initial issue! For testing with the COCO test set and generating JSON files, you can follow these steps:
Testing with COCO Test Set
-
Prepare the COCO Test Set: Ensure you have the COCO test images and the corresponding annotations in the correct format.
-
Modify the YAML Configuration: Update your dataset YAML file to include the path to the test images. Here’s an example:
path: ../datasets/coco train: images/train val: images/val test: images/test # Add this line for test images kpt_shape: [17, 3] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible) flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15] names: 0: person -
Run the Validation: Use the following command to run the validation on the test set:
yolo pose val model=path/to/best.pt data=coco-pose.yaml split=test save_json=True
Generating JSON Files for COCO Evaluation
To generate the COCO-format JSON predictions, set
save_json=Truewhen validating on the test split:from ultralytics import YOLO model = YOLO("path/to/best.pt") metrics = model.val(data="coco-pose.yaml", split="test", save_json=True)This writes a
predictions.jsonfile with image IDs, category IDs, boxes, scores and keypoints in COCO format to the validation run directory (for exampleruns/pose/val), ready for the evaluation server. Note thatmodel.val()returns a metrics object, not a list of results, so it cannot be iterated.Additional Resources
For more detailed information, you can refer to the Pose Estimation Datasets Overview. This page provides comprehensive details on dataset formats, YAML configuration, and supported datasets.
If you encounter any issues or need further assistance, please provide a reproducible example of your problem. This helps us diagnose and resolve the issue more effectively. You can find guidance on creating a minimum reproducible example here.
Feel free to ask if you have more questions! 😊
-
1-zym
I have the following error according to the way you provided. In addition, can you provide the detailed configuration of pose training? For example: epoch, lr, whether pretrain, etc., the yolom-pose results of training 300 epoches by myself have a big gap with the results provided Ultralytics YOLOv8.2.54 🚀 Python-3.8.19 torch-1.12.1+cu116 CUDA:0 (NVIDIA A40, 45449MiB) YOLOv8m-pose summary (fused): 237 layers, 26447596 parameters, 0 gradients, 81.0 GFLOPs val: Scanning /opt/storage/zhaoyiming/datasets/coco-pose/labels/test2017.cache.. WARNING ⚠️ No labels found in /opt/storage/zhaoyiming/datasets/coco-pose/labels/test2017.cache, training may not work correctly. See https://docs.ultralytics.com/datasets for dataset formatting guidance. Class Images Instances Box(P R mAP50 m all 40670 0 0 0 0 0 0 0 0 0 WARNING ⚠️ no labels found in pose set, can not compute metrics without labels Speed: 0.1ms preprocess, 4.1ms inference, 0.0ms loss, 0.5ms postprocess per image Results saved to /opt/storage/zhaoyiming/ultralytics/runs/pose/val14 Traceback (most recent call last): File "/opt/storage/zhaoyiming/ultralytics/ultralytics/utils/test.py", line 13, in for result in results: TypeError: 'PoseMetrics' object is not iterable
Process finished with exit code 1
Glenn JocherUltralytics(수정됨)Hi @1-zym, thanks for reaching out! It looks like you're encountering an issue with missing labels during validation and a
TypeErrorin your script. Here are a few steps to help resolve these issues:Missing Labels Warning
The warning indicates that no labels were found in your test set. Ensure that your dataset is correctly formatted and that the labels are in the expected location. You can refer to the dataset formatting guidance for more details.
Configuration for Pose Training
Here's a detailed example configuration for training a YOLO26-Pose model:
from ultralytics import YOLO # Load a pretrained model model = YOLO("yolo26m-pose.pt") # Train the model results = model.train( data="coco-pose.yaml", # Path to your dataset YAML file epochs=300, # Number of epochs imgsz=640, # Image size lr0=0.01, # Initial learning rate pretrained=True # Use pretrained weights )Generating JSON Files for COCO Evaluation
To generate the COCO-format JSON predictions, set
save_json=Truewhen validating on the test split:from ultralytics import YOLO model = YOLO("path/to/best.pt") metrics = model.val(data="coco-pose.yaml", split="test", save_json=True)This writes a
predictions.jsonfile with image IDs, category IDs, boxes, scores and keypoints in COCO format to the validation run directory (for exampleruns/pose/val), ready for the evaluation server. Note thatmodel.val()returns a metrics object, not a list of results, so it cannot be iterated.TypeError: 'PoseMetrics' object is not iterable
model.val()returns a metrics object, not a list of results, so it cannot be iterated; usesave_json=Trueas shown above. Ensure you are using the latest version of the Ultralytics package and that your script aligns with the expected API.If the issue persists, please provide a minimum reproducible example to help us diagnose the problem more effectively. You can find guidance on creating a reproducible example here.
Feel free to ask if you have more questions! 😊
1-zym
path: ../datasets/coco-pose train: train2017.txt val: val2017.txt test: test-dev2017.txt, 通过coco-pose.yaml,我并没有在我的文件夹成功下载 test-dev2017.txt,是官方没有提供?需要我自己生成? 可以提供一下yolov8-pose的详细训练配置?我自己训练300epoch的yolov8m-pose与你们提供的yolov8m-pose的精度,在coco-pose数据集上map50-90差了将近10个点
1-zym
0 0.620039 0.5939 0.172415 0.14608 0.658793 0.552 2 0.664042 0.55 2 0.656168 0.548 2 0.67979 0.554 2 0 0 0 0.67979 0.57 2 0.648294 0.568 2 0.674541 0.588 2 0.606299 0.582 2 0.627297 0.562 2 0.603675 0.564 2 0.627297 0.616 2 0 0 0 0.595801 0.604 2 0 0 0 0.569554 0.644 2 0 0 0 0 0.385525 0.58557 0.14937 0.12586 0.414698 0.54 1 0.419948 0.538 2 0.412073 0.538 1 0.43832 0.54 2 0 0 0 0.435696 0.558 2 0.417323 0.554 2 0.396325 0.574 2 0 0 0 0 0 0 0 0 0 0.414698 0.608 2 0.39895 0.604 2 0.372703 0.592 2 0 0 0 0.346457 0.638 2 0 0 0 0 0.328898 0.70199 0.0313386 0.06714 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.760577 0.69423 0.0285564 0.05486 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.510709 0.69215 0.0187927 0.04682 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.929554 0.67602 0.0388451 0.01844 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 上面的是我自己通过脚本将coco 关键点标注文件转换为yolo 格式 0 0.620039 0.593900 0.172415 0.146080 0.658793 0.552000 2.000000 0.664042 0.550000 2.000000 0.656168 0.548000 2.000000 0.679790 0.554000 2.000000 0.000000 0.000000 0.000000 0.679790 0.570000 2.000000 0.648294 0.568000 2.000000 0.674541 0.588000 2.000000 0.606299 0.582000 2.000000 0.627297 0.562000 2.000000 0.603675 0.564000 2.000000 0.627297 0.616000 2.000000 0.000000 0.000000 0.000000 0.595801 0.604000 2.000000 0.000000 0.000000 0.000000 0.569554 0.644000 2.000000 0.000000 0.000000 0.000000 0 0.385525 0.585570 0.149370 0.125860 0.414698 0.540000 1.000000 0.419948 0.538000 2.000000 0.412073 0.538000 1.000000 0.438320 0.540000 2.000000 0.000000 0.000000 0.000000 0.435696 0.558000 2.000000 0.417323 0.554000 2.000000 0.396325 0.574000 2.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.414698 0.608000 2.000000 0.398950 0.604000 2.000000 0.372703 0.592000 2.000000 0.000000 0.000000 0.000000 0.346457 0.638000 2.000000 0.000000 0.000000 0.000000 这是我通过coco-pose.yaml下载的,用上面的生成的会对训练结果造成影响?
Glenn JocherUltralytics(수정됨)Hi @1-zym, thank you for your detailed message. It looks like you're encountering a few issues related to dataset preparation and training configuration. Let's address them step-by-step.
Missing
test-dev2017.txtThe
test-dev2017.txtfile is not provided by default. You need to generate this file yourself by listing the paths to your test images. Here’s a quick way to create it:find /path/to/test/images -type f -name "*.jpg" > test-dev2017.txtDetailed Training Configuration for YOLO26-Pose
To achieve optimal results, it's crucial to fine-tune your training configuration. Here’s an example configuration for training YOLO26m-Pose:
from ultralytics import YOLO # Load a pretrained model model = YOLO("yolo26m-pose.pt") # Train the model results = model.train( data="coco-pose.yaml", # Path to your dataset YAML file epochs=300, # Number of epochs imgsz=640, # Image size lr0=0.01, # Initial learning rate pretrained=True, # Use pretrained weights batch=16, # Batch size optimizer='Adam', # Optimizer patience=50, # Early stopping patience )Generating JSON Files for COCO Evaluation
To generate the COCO-format JSON predictions, set
save_json=Truewhen validating on the test split:from ultralytics import YOLO model = YOLO("path/to/best.pt") metrics = model.val(data="coco-pose.yaml", split="test", save_json=True)This writes a
predictions.jsonfile with image IDs, category IDs, boxes, scores and keypoints in COCO format to the validation run directory (for exampleruns/pose/val), ready for the evaluation server. Note thatmodel.val()returns a metrics object, not a list of results, so it cannot be iterated.Addressing the TypeError
model.val()returns a metrics object, not a list of results, so it cannot be iterated; usesave_json=Trueas shown above. Ensure you are using the latest version of the Ultralytics package. If the issue persists, please provide a minimum reproducible example to help us diagnose the problem more effectively. You can find guidance on creating a reproducible example here.Feel free to ask if you have more questions! 😊
1-zym
现在我遇到一个问题,我用yolov8m-pose使用相同的训练参数、环境,只是GPU不同,训练300个epoch,不使用预训练,其他参数都是默认,我的精度比他的低了10个点,此外我在自己的卡上使用100个epoch训练yolov8n-pose比使用300个epoch 精度也高了7,8个点,你能为我解答这个问题?
1-zym
task: pose mode: train model: ultralytics/cfg/models/v8/yolov8-pose.yaml data: ultralytics/cfg/datasets/coco-pose.yaml epochs: 300 time: null patience: 100 batch: 64 imgsz: 800 save: true save_period: -1 cache: true device: '1' workers: 8 project: null name: train35 exist_ok: false pretrained: false optimizer: SGD verbose: true seed: 0 deterministic: true single_cls: false rect: false cos_lr: false close_mosaic: 10 resume: false amp: true fraction: 1.0 profile: false freeze: null multi_scale: false overlap_mask: true mask_ratio: 4 dropout: 0.0 val: true split: val save_json: false save_hybrid: false conf: null iou: 0.7 max_det: 300 half: false dnn: false plots: true source: null vid_stride: 1 stream_buffer: false visualize: false augment: false agnostic_nms: false classes: null retina_masks: false embed: null show: false save_frames: false save_txt: false save_conf: false save_crop: false show_labels: true show_conf: true show_boxes: true line_width: null format: torchscript keras: false optimize: false int8: false dynamic: false simplify: false opset: null workspace: 4 nms: false lr0: 0.01 lrf: 0.01 momentum: 0.937 weight_decay: 0.0005 warmup_epochs: 3.0 warmup_momentum: 0.8 warmup_bias_lr: 0.1 box: 7.5 cls: 0.5 dfl: 1.5 pose: 12.0 kobj: 1.0 label_smoothing: 0.0 nbs: 64 hsv_h: 0.015 hsv_s: 0.7 hsv_v: 0.4 degrees: 0.0 translate: 0.1 scale: 0.5 shear: 0.0 perspective: 0.0 flipud: 0.0 fliplr: 0.5 bgr: 0.0 mosaic: 1.0 mixup: 0.0 copy_paste: 0.0 auto_augment: randaugment erasing: 0.4 crop_fraction: 1.0 cfg: null tracker: botsort.yaml save_dir: /opt/storage/zhaoyiming/ultralytics/runs/pose/train35 这是参数文件
Glenn JocherUltralyticsIt sounds like you're experiencing significant variations in performance despite using similar training parameters. Differences in GPU architecture can sometimes lead to variations in training results due to factors like hardware-specific optimizations and floating-point precision. Additionally, training stability and convergence can be influenced by random initialization and data augmentation. To address this, ensure you're using the latest version of the Ultralytics package and try setting a fixed random seed for reproducibility. If the issue persists, consider experimenting with different learning rates, batch sizes, and optimizers to find the optimal configuration for your specific setup.
1-zym
为什么yolov8n-pose训练100个epoch会比300个epoch要好呢?lr0=0.01 lrf=0.01 ,其他的超参数都一样,都是默认,我不是很明白它的学习率是怎么衰减的。
Glenn JocherUltralyticsThe difference in performance between training for 100 epochs and 300 epochs could be due to overfitting. With a learning rate of 0.01 and default parameters, the model might be learning too quickly and overfitting to the training data. To address this, consider using a learning rate scheduler or early stopping. Additionally, ensure that your dataset is well-augmented and that the training process is consistent across different GPUs. For more details on learning rate schedules and training strategies, you can refer to the Ultralytics documentation.
