이미지 분류 데이터셋 개요
YOLO 분류 작업을 위한 데이터셋 구조#
Ultralytics YOLO 분류 작업에서는 root 디렉터리 아래에 데이터셋을 특정한 분할 디렉터리 구조로 구성해야 합니다. 이 구조에는 train 디렉터리와 val 디렉터리(valid 및 validation도 허용됨)가 포함되며, test 디렉터리는 선택 사항입니다. 검증 디렉터리가 없으면 test 디렉터리를 대신 검증에 사용합니다. 루트에 train 디렉터리가 전혀 없지만 클래스별 폴더가 하나씩 있는 경우, Ultralytics는 이미지를 80/20 비율로 자동 분할하여 <root>_split 디렉터리를 새로 만들고 그 안에 train 및 val 하위 디렉터리를 생성합니다.
각 디렉터리에는 데이터셋의 각 클래스에 해당하는 하위 디렉터리가 하나씩 있어야 합니다. 하위 디렉터리의 이름은 해당 클래스에 따라 지정하며, 해당 클래스의 모든 이미지를 포함합니다. 각 이미지 파일의 이름이 고유하고 JPEG 또는 PNG와 같은 일반적인 형식으로 저장되어 있는지 확인하세요.
폴더 구조 예시#
CIFAR-10 데이터셋을 예로 들어 보겠습니다. 폴더 구조는 다음과 같아야 합니다.
cifar10/
|
|-- train/
| |-- airplane/
| | |-- 10008_airplane.png
| | |-- 10009_airplane.png
| | |-- ...
| |
| |-- automobile/
| | |-- 1000_automobile.png
| | |-- 1001_automobile.png
| | |-- ...
| |
| |-- bird/
| | |-- 10014_bird.png
| | |-- 10015_bird.png
| | |-- ...
| |
| |-- ...
|
|-- test/ (optional)
| |-- airplane/
| | |-- 10_airplane.png
| | |-- 11_airplane.png
| | |-- ...
| |
| |-- automobile/
| | |-- 100_automobile.png
| | |-- 101_automobile.png
| | |-- ...
| |
| |-- bird/
| | |-- 1000_bird.png
| | |-- 1001_bird.png
| | |-- ...
| |
| |-- ...
|
|-- val/
| |-- airplane/
| | |-- 105_airplane.png
| | |-- 106_airplane.png
| | |-- ...
| |
| |-- automobile/
| | |-- 102_automobile.png
| | |-- 103_automobile.png
| | |-- ...
| |
| |-- bird/
| | |-- 1045_bird.png
| | |-- 1046_bird.png
| | |-- ...
| |
| |-- ...이와 같이 구조화하면 학습 단계에서 모델이 잘 정리된 클래스를 효과적으로 학습하고 테스트 및 검증 단계에서 성능을 정확하게 평가할 수 있습니다.
사용법#
from ultralytics import YOLO
# 모델 로드
model = YOLO("yolo26n-cls.pt") # 사전 학습된 모델을 불러옵니다(학습에 권장).
# 모델 학습
results = model.train(data="path/to/dataset", epochs=100, imgsz=640)대부분의 기본 제공 데이터셋 이름(예: cifar10, imagenette 또는 mnist160)은 처음 참조할 때 데이터를 자동으로 다운로드하고 캐시합니다. 직접 선별한 사용자 지정 데이터셋이 있는 경우에만 data을 폴더 경로로 지정하세요.
지원되는 데이터셋#
Ultralytics는 다음 데이터셋을 자동으로 다운로드하도록 지원합니다. 이 데이터셋 대부분은 Ultralytics Platform에도 호스팅되어 있어 이미지와 어노테이션을 살펴보고, 데이터셋 통계를 확인하고, 클라우드 학습을 위해 데이터셋을 복제할 수 있습니다.
- Caltech 101: 이미지 분류 작업용 객체 범주 101개의 이미지가 포함된 데이터셋입니다.
- Caltech 256: 객체 범주 256개와 더 까다로운 이미지를 포함하는 Caltech 101의 확장 버전입니다.
- CIFAR-10: 10개 클래스의 32x32 컬러 이미지 6만 개로 구성되며, 클래스마다 이미지 6천 개를 포함하는 데이터셋입니다.
- CIFAR-100: 객체 범주 100개와 클래스당 이미지 600개를 포함하는 CIFAR-10의 확장 버전입니다.
- Fashion-MNIST: 이미지 분류 작업을 위해 패션 범주 10개의 그레이스케일 이미지 7만 장으로 구성된 데이터셋입니다.
- ImageNet: 클래스 1,000개, 학습 이미지 128만 개, 검증 이미지 50,000개로 구성된 ImageNet-1k (ILSVRC-2012) 이미지 분류 벤치마크입니다.
- ImageNet-10: 더 빠른 실험과 테스트를 위해 범주 10개로 구성한 ImageNet의 소규모 하위 집합입니다.
- Imagenette: 빠른 학습과 테스트를 위해 쉽게 구분할 수 있는 클래스 10개를 포함한 ImageNet의 소규모 하위 집합입니다.
- Imagewoof: 이미지 분류 작업을 위해 개 품종 범주 10개를 포함하는 더 까다로운 ImageNet 하위 집합입니다.
- MNIST: 이미지 분류 작업을 위한 손글씨 숫자 그레이스케일 이미지 7만 장으로 구성된 데이터셋입니다.
- MNIST160: MNIST 학습 및 테스트 분할에서 숫자(0~9)마다 처음 8개 이미지를 가져온 데이터셋으로, 총 160개 이미지로 구성됩니다.
사용자 데이터셋 추가#
자체 데이터셋을 사용해 Ultralytics YOLO로 분류 모델을 학습하려면 먼저 위의 "데이터셋 구조"에 지정된 형식을 따르는지 확인한 다음, 학습 스크립트를 초기화할 때 data 인수가 데이터셋 디렉터리를 가리키도록 설정하세요.
자주 묻는 질문#
Ultralytics YOLO 분류 작업용 데이터셋을 구성하려면 특정한 분할 디렉터리 형식을 따라야 합니다. 데이터셋을
train및val디렉터리로 나누고, 필요한 경우test디렉터리도 추가하세요. 각 디렉터리에는 클래스별 이름을 가진 하위 디렉터리가 있어야 하며, 해당 클래스의 이미지가 그 안에 있어야 합니다. 이렇게 구성하면 원활한 학습 및 평가가 가능합니다. 예를 들어 CIFAR-10 데이터셋 형식을 살펴보세요.cifar10/ |-- train/ | |-- airplane/ | |-- automobile/ | |-- bird/ | ... |-- val/ | |-- airplane/ | |-- automobile/ | |-- bird/ | ... |-- test/ (optional) | |-- airplane/ | |-- automobile/ | |-- bird/ | ...자세한 내용은 YOLO 분류 작업을 위한 데이터셋 구조 섹션을 참조하세요.
Ultralytics YOLO는 이미지 분류를 위해 Caltech 101, Caltech 256, CIFAR-10, CIFAR-100, Fashion-MNIST, ImageNet, ImageNet-10, Imagenette, Imagewoof, MNIST 등 여러 데이터셋의 자동 다운로드를 지원합니다. 이러한 데이터셋은 YOLO에서 쉽게 사용할 수 있도록 구성되어 있습니다. 각 데이터셋 페이지에서 구조와 애플리케이션에 관한 자세한 내용을 확인할 수 있습니다.
Ultralytics YOLO에서 자체 데이터셋을 사용하려면 분류 작업에 필요한 지정된 디렉터리 형식을 따르는지 확인하세요. 별도의
train및val디렉터리(선택적으로test포함)와 각 클래스의 해당 이미지가 들어 있는 하위 디렉터리가 필요합니다. 데이터셋을 올바르게 구성한 다음, 학습 스크립트를 초기화할 때data인수가 데이터셋의 루트 디렉터리를 가리키도록 설정하세요. 다음은 Python 예시입니다.from ultralytics import YOLO # 모델 로드 model = YOLO("yolo26n-cls.pt") # 사전 학습된 모델을 불러옵니다(학습에 권장). # 모델 학습 results = model.train(data="path/to/your/dataset", epochs=100, imgsz=640)자세한 내용은 자체 데이터셋 추가 섹션에서 확인할 수 있습니다.
Ultralytics YOLO는 이미지 분류에 다음과 같은 여러 이점을 제공합니다.
- 사전 학습 모델:
yolo26n-cls.pt과 같은 사전 학습 모델을 불러와 학습을 빠르게 시작할 수 있습니다. - 사용 편의성: 학습 및 평가를 위한 간단한 API 및 CLI 명령을 제공합니다.
- 높은 성능: 우수한 정확도와 속도를 제공하여 실시간 애플리케이션에 적합합니다.
- 다중 데이터셋 지원: CIFAR-10, ImageNet 등 널리 사용되는 데이터셋을 기본 지원합니다.
- 커뮤니티 및 지원: 문제 해결과 개선을 위한 광범위한 문서와 활발한 커뮤니티를 이용할 수 있습니다.
추가 정보와 실제 애플리케이션은 Ultralytics YOLO에서 살펴볼 수 있습니다.
- 사전 학습 모델:
Python 또는 CLI에서 Ultralytics YOLO를 사용해 모델을 학습할 수 있습니다. 예시는 다음과 같습니다:
예제from ultralytics import YOLO # 모델 로드 model = YOLO("yolo26n-cls.pt") # 사전 학습 모델 불러오기 # 모델 학습 results = model.train(data="path/to/dataset", epochs=100, imgsz=640)이 예시는 두 가지 방식 중 하나를 사용해 YOLO 모델을 간편하게 학습하는 과정을 보여줍니다. 자세한 내용은 사용법 섹션과 분류 작업용 학습 페이지를 참조하세요.
댓글45
질문을 하거나 결과를 공유하거나 다른 사람을 도와주세요.
iongion
My annotations have regions that I know variate a lot and cannot be used for classification. Is there a way to feed these regions so that it avoids learning from them during training ? If I mask/fill the areas with black color on the image for training itself, wouldn't that affect classification ? Sometimes some images might have black shapes as features themselves.
lauravchrobak
Its confusing to me that the val set is optional for classification but the test set is optional for detection. Should there be some consistency there?
Glenn JocherUltralytics(수정됨)@lauravchrobak great observation! For classification,
trainplus either avalor atestfolder is required: if one ofval/testis missing, Ultralytics uses the other for validation. Detection datasets use a YAML file wheretrainandvalare required andtestis optional. Both layouts are described in the classification and detection dataset docs.
1ZZZyyyh
that's why? I've made my dataset as the sample.
File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/engine/trainer.py", line 567, in get_dataset raise RuntimeError(emojis(f"Dataset '{clean_url(self.args.data)}' error ❌ {e}")) from e RuntimeError: Dataset '/home/visitor/yolov8-tb11' error ❌ [Errno 21] Is a directory: '/home/visitor/yolov8-tb11'
Glenn JocherUltralytics(수정됨)This error appears when a directory is passed as
datato a non-classification task. For classification,datashould be the dataset root directory (withtrain/andval/ortest/folders, one subfolder per class), and you should load a classification model such asyolo26n-cls.pt(or settask=classify). For detection, segmentation, pose or OBB,datamust be adata.yamlfile. See Dataset Structure for YOLO Classification Tasks.1ZZZyyyh
I want to train my model , it reports error. yolo task=classify mode=train model=yolov8m-cls.pt data=/home/visitor/yolov8-tb11 epochs=100 imgsz=640 ... Traceback (most recent call last): File "/home/visitor/anaconda3/bin/yolo", line 8, in sys.exit(entrypoint()) ^^^^^^^^^^^^ File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/cfg/init.py", line 991, in entrypoint getattr(model, mode)(**overrides) # default args from model ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/engine/model.py", line 806, in train self.trainer.train() File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/engine/trainer.py", line 207, in train self._do_train(world_size) File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/engine/trainer.py", line 323, in _do_train self._setup_train(world_size) File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/engine/trainer.py", line 264, in _setup_train self.amp = torch.tensor(check_amp(self.model), device=self.device) ^^^^^^^^^^^^^^^^^^^^^ File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/utils/checks.py", line 707, in check_amp assert amp_allclose(YOLO("yolo11n.pt"), im) ^^^^^^^^^^^^^^^^^^ File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/models/yolo/model.py", line 23, in init super().init(model=model, task=task, verbose=verbose) File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/engine/model.py", line 146, in init self._load(model, task=task) File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/engine/model.py", line 289, in _load self.model, self.ckpt = attempt_load_one_weight(weights) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/nn/tasks.py", line 908, in attempt_load_one_weight ckpt, weight = torch_safe_load(weight) # load ckpt ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/nn/tasks.py", line 835, in torch_safe_load ckpt = torch.load(file, map_location="cpu") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/visitor/anaconda3/lib/python3.12/site-packages/ultralytics/utils/patches.py", line 86, in torch_load return _torch_load(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/visitor/anaconda3/lib/python3.12/site-packages/torch/serialization.py", line 1326, in load with _open_zipfile_reader(opened_file) as opened_zipfile: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/visitor/anaconda3/lib/python3.12/site-packages/torch/serialization.py", line 671, in init super().init(torch._C.PyTorchFileReader(name_or_buffer)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
Glenn JocherUltralytics@1ZZZyyyh the error indicates an issue with your dataset path or structure. Ensure the
dataargument points to a valid dataset configuration file (.yaml) or a properly structured classification dataset withtrain/,test/, and optionallyval/directories, as outlined in the classification dataset structure guide. Let me know if you need further assistance!
rishabh-jain14
I have this yaml file for caltech-256 classification: path: D:/Coursework/Fall 24/CSC591-007 DLBA/Project/Project/data/tvt_Caltech_256 train: train/ val: val/ test: test/ nc: 257 names: ['001.ak47', '002.american-flag', '003.backpack', '004.baseball-bat', '005.baseball-glove', '006.basketball-hoop', '007.bat', '008.bathtub', '009.bear', '010.beer-mug', '011.billiards', '012.binoculars', '013.birdbath', '014.blimp', '015.bonsai-101', '016.boom-box', '017.bowling-ball', '018.bowling-pin', '019.boxing-glove', '020.brain-101', '021.breadmaker', '022.buddha-101', '023.bulldozer', '024.butterfly', '025.cactus', '026.cake', '027.calculator', '028.camel', '029.cannon', '030.canoe', '031.car-tire', '032.cartman', '033.cd', '034.centipede', '035.cereal-box', '036.chandelier-101', '037.chess-board', '038.chimp', '039.chopsticks', '040.cockroach', '041.coffee-mug', '042.coffin', '043.coin', '044.comet', '045.computer-keyboard', '046.computer-monitor', '047.computer-mouse', '048.conch', '049.cormorant', '050.covered-wagon', '051.cowboy-hat', '052.crab-101', '053.desk-globe', '054.diamond-ring', '055.dice', '056.dog', '057.dolphin-101', '058.doorknob', '059.drinking-straw', '060.duck', '061.dumb-bell', '062.eiffel-tower', '063.electric-guitar-101', '064.elephant-101', '065.elk', '066.ewer-101', '067.eyeglasses', '068.fern', '069.fighter-jet', '070.fire-extinguisher', '071.fire-hydrant', '072.fire-truck', '073.fireworks', '074.flashlight', '075.floppy-disk', '076.football-helmet', '077.french-horn', '078.fried-egg', '079.frisbee', '080.frog', '081.frying-pan', '082.galaxy', '083.gas-pump', '084.giraffe', '085.goat', '086.golden-gate-bridge', '087.goldfish', '088.golf-ball', '089.goose', '090.gorilla', '091.grand-piano-101', '092.grapes', '093.grasshopper', '094.guitar-pick', '095.hamburger', '096.hammock', '097.harmonica', '098.harp', '099.harpsichord', '100.hawksbill-101', '101.head-phones', '102.helicopter-101', '103.hibiscus', '104.homer-simpson', '105.horse', '106.horseshoe-crab', '107.hot-air-balloon', '108.hot-dog', '109.hot-tub', '110.hourglass', '111.house-fly', '112.human-skeleton', '113.hummingbird', '114.ibis-101', '115.ice-cream-cone', '116.iguana', '117.ipod', '118.iris', '119.jesus-christ', '120.joy-stick', '121.kangaroo-101', '122.kayak', '123.ketch-101', '124.killer-whale', '125.knife', '126.ladder', '127.laptop-101', '128.lathe', '129.leopards-101', '130.license-plate', '131.lightbulb', '132.light-house', '133.lightning', '134.llama-101', '135.mailbox', '136.mandolin', '137.mars', '138.mattress', '139.megaphone', '140.menorah-101', '141.microscope', '142.microwave', '143.minaret', '144.minotaur', '145.motorbikes-101', '146.mountain-bike', '147.mushroom', '148.mussels', '149.necktie', '150.octopus', '151.ostrich', '152.owl', '153.palm-pilot', '154.palm-tree', '155.paperclip', '156.paper-shredder', '157.pci-card', '158.penguin', '159.people', '160.pez-dispenser', '161.photocopier', '162.picnic-table', '163.playing-card', '164.porcupine', '165.pram', '166.praying-mantis', '167.pyramid', '168.raccoon', '169.radio-telescope', '170.rainbow', '171.refrigerator', '172.revolver-101', '173.rifle', '174.rotary-phone', '175.roulette-wheel', '176.saddle', '177.saturn', '178.school-bus', '179.scorpion-101', '180.screwdriver', '181.segway', '182.self-propelled-lawn-mower', '183.sextant', '184.sheet-music', '185.skateboard', '186.skunk', '187.skyscraper', '188.smokestack', '189.snail', '190.snake', '191.sneaker', '192.snowmobile', '193.soccer-ball', '194.socks', '195.soda-can', '196.spaghetti', '197.speed-boat', '198.spider', '199.spoon', '200.stained-glass', '201.starfish-101', '202.steering-wheel', '203.stirrups', '204.sunflower-101', '205.superman', '206.sushi', '207.swan', '208.swiss-army-knife', '209.sword', '210.syringe', '211.tambourine', '212.teapot', '213.teddy-bear', '214.teepee', '215.telephone-box', '216.tennis-ball', '217.tennis-court', '218.tennis-racket', '219.theodolite', '220.toaster', '221.tomato', '222.tombstone', '223.top-hat', '224.touring-bike', '225.tower-pisa', '226.traffic-light', '227.treadmill', '228.triceratops', '229.tricycle', '230.trilobite-101', '231.tripod', '232.t-shirt', '233.tuning-fork', '234.tweezer', '235.umbrella-101', '236.unicorn', '237.vcr', '238.video-projector', '239.washing-machine', '240.watch-101', '241.waterfall', '242.watermelon', '243.welding-mask', '244.wheelbarrow', '245.windmill', '246.wine-bottle', '247.xylophone', '248.yarmulke', '249.yo-yo', '250.zebra', '251.airplanes-101', '252.car-side-101', '253.faces-easy-101', '254.greyhound', '255.tennis-shoes', '256.toad', '257.clutter']
However, when I train, it shows: train: Scanning D:\Coursework\Fall 24\CSC591-007 DLBA\Project\Project\data\tvt_Caltech_256\train\001... 0 images, 24485 backgrounds, 0 corrupt: 100%|██████████| 24485/24485 [00:43<00:00, 565.92it/s] train: WARNING No labels found in D:\Coursework\Fall 24\CSC591-007 DLBA\Project\Project\data\tvt_Caltech_256\train\001.cache. See https://docs.ultralytics.com/datasets for dataset formatting guidance.
train: New cache created: D:\Coursework\Fall 24\CSC591-007 DLBA\Project\Project\data\tvt_Caltech_256\train\001.cache WARNING No labels found in D:\Coursework\Fall 24\CSC591-007 DLBA\Project\Project\data\tvt_Caltech_256\train\001.cache, training may not work correctly. See https://docs.ultralytics.com/datasets for dataset formatting guidance. val: Scanning D:\Coursework\Fall 24\CSC591-007 DLBA\Project\Project\data\tvt_Caltech_256\val\001... 0 images, 3061 backgrounds, 0 corrupt: 100%|██████████| 3061/3061 [00:02<00:00, 1231.29it/s] val: WARNING No labels found in D:\Coursework\Fall 24\CSC591-007 DLBA\Project\Project\data\tvt_Caltech_256\val\001.cache. See https://docs.ultralytics.com/datasets for dataset formatting guidance.
val: New cache created: D:\Coursework\Fall 24\CSC591-007 DLBA\Project\Project\data\tvt_Caltech_256\val\001.cache WARNING No labels found in D:\Coursework\Fall 24\CSC591-007 DLBA\Project\Project\data\tvt_Caltech_256\val\001.cache, training may not work correctly. See https://docs.ultralytics.com/datasets for dataset formatting guidance. Plotting labels to runs\detect\train\labels.jpg... zero-size array to reduction operation maximum which has no identity optimizer: 'optimizer=auto' found, ignoring 'lr0=0.01' and 'momentum=0.937' and determining best 'optimizer', 'lr0' and 'momentum' automatically... optimizer: AdamW(lr=3.8e-05, momentum=0.9) with parameter groups 81 weight(decay=0.0), 88 weight(decay=0.0005), 87 bias(decay=0.0) TensorBoard: model graph visualization added Image sizes 128 train, 128 val
What should I do to resolve the issue?
Glenn JocherUltralytics@rishabh-jain14 it looks like the issue stems from the dataset structure not aligning with the expected format for classification tasks. Each class should have its own directory containing the respective images. Please ensure your dataset follows this structure. For detailed guidance, refer to the Image Classification Datasets Overview.
NajeebAhmad00
I am using a yolov8-cls.pt model and want to add a new class to it. I need to train the model on this new class without affecting the existing classes. How can I do this?
Glenn JocherUltralytics(수정됨)@NajeebAhmad00 Ultralytics classification models cannot add a class without retraining: the classifier head is rebuilt for the classes in your dataset. If you fine-tune only on images of the new class, the model will forget the existing ones. To keep the existing classes, include their images (or a representative sample) along with the new class in your dataset, organized as described in the Ultralytics documentation, then fine-tune:
from ultralytics import YOLO # Load the pretrained model model = YOLO("yolo26n-cls.pt") # Train on the dataset containing the old and new classes results = model.train(data="path/to/your/dataset", epochs=100, imgsz=224)
leolle520
Can I use this model to train an one class image classifier as an abnormal image detector? Thank you.
Glenn JocherUltralyticsYes, you can use the YOLO model to train a one-class image classifier for abnormal image detection. Simply organize your dataset with one class for normal images and another for abnormal images, then proceed with training. For more details on dataset structure, visit our documentation.
MrLi-creator
This is my own small_data.yaml file: path: C:/Users/AIX1WX/Documents/10-POC/train_yolov8/yolov8-all-in-one-main_classification/datasets train: train/ val: val/ test: test/
nc: 7 names: ['ABS','Airbag','Background','EPB','EPBGREEN','EPBRed','QMtt']
class_weights: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 3.0]
And my train coda here: from ultralytics import YOLO import os os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE" if name == 'main': model = YOLO("yolov8n-cls.pt") # load a pretrained model (recommended for training) results = model.train(data="small_data.yaml", epochs=30, imgsz=50, device=None, batch=16)
Why will report the error: Dataset not found , missing path C:\Users\AIX1WX\Documents\10-POC\train_yolov8\yolov8-all-in-one-main_detection\datasets\small_data.yaml, attempting download... Downloading https://github.com/ultralytics/yolov5/releases/download/v1.0/small_data.yaml.zip to 'C:\Users\AIX1WX\Documents\10-POC\train_yolov8\yolov8-all-in-one-main_detection\datasets\small_data.yaml.zip'...
Glenn JocherUltralytics(수정됨)@MrLi-creator the error occurs because classification does not use a YAML file. Pass the dataset root directory as
data(for exampledata="C:/Users/AIX1WX/Documents/10-POC/train_yolov8/yolov8-all-in-one-main_classification/datasets"), containingtrain/,val/and optionallytest/folders, each with one subfolder per class. Keys such asnc,namesandclass_weightsin your YAML are ignored. For more details on dataset structure, refer to the Ultralytics documentation.MrLi-creator
Hi glenn-jocher, I find the case: I first train yolov8n.pt in yolov8-all-in-one-main_detection file. And second time I train yolov8n-cls.pt in another file yolov8-all-in-one-main_classification. Then everytime train yolov8n-cls.pt, the dataset path always points to yolov8-all-in-one-main_detection. So is there any way to change the default path of the dataset
Glenn JocherUltralytics(수정됨)@MrLi-creator it seems the issue is that
small_data.yamlis not a valid classification dataset. Pass the dataset root directory (withtrain/,val/and optionallytest/folders, each containing one subfolder per class) asdatainstead of a YAML file. For further details on dataset structure, you can refer to the Ultralytics documentation.MrLi-creator
Hi glenn-jocher, I just find that the classification task can't use data.yaml. You can see pderrenger comment in the link(https://github.com/ultralytics/ultralytics/issues/13789)
Glenn JocherUltralyticsThank you for pointing that out. For classification tasks with YOLO, the dataset should be organized in a specific directory structure rather than using a
data.yamlfile. Ensure your dataset follows the format with separatetrain,val, andtestdirectories, each containing subdirectories for each class. This structure is essential for proper training and evaluation. For more details, you can refer to the Ultralytics documentation on dataset structure.
MrLi-creator
Hi, how can I change the class weight in the training of yolov8n-cls.pt?
Paula DerrengerUltralytics(수정됨)@MrLi-creator Ultralytics does not have a
class_weightstraining argument, so there is no setting to change class weights when trainingyolo26n-cls.pt. To handle class imbalance in classification, balance your dataset (for example by oversampling the minority classes) or modify the loss in the source code. See the classification dataset guide for the expected dataset structure.MrLi-creator
Hi pderrenger, Firstly, thank you for your response. I am trying to add classw_weights to the data.yaml file. However, it was found that the classification task cannot use the data.yaml file, as you suggested in the link(https://github.com/ultralytics/ultralytics/issues/13789). So are there any other ways to modify classw_weights in classification, preferably with an example
Paula DerrengerUltralytics(수정됨)@MrLi-creator Ultralytics does not support a
class_weightsparameter for classification training, and adata.yamlfile isn't used for classification tasks (you pass the dataset directory asdata). To handle class imbalance, balance your dataset or modify the loss function in the source. See the Ultralytics documentation on classification datasets.MrLi-creator
@pderrenger Hi pderrenger. I have set
class_weightsparameter in the training script. But it report error.results = model.train(data='./datasets', epochs=30, imgsz=50, device=None, batch=16,classes =[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 3.0] ) SyntaxError: 'class_weights' is not a valid YOLO argument. Similar arguments are i.e. ['classes'].
Paula DerrengerUltralyticsHi @MrLi-creator,
The
class_weightsparameter is not directly supported in thetrainmethod for YOLO classification tasks. Instead, you can handle class imbalance by adjusting the dataset or using techniques like data augmentation. If you encounter any issues, please ensure you are using the latest version of the Ultralytics package. For more detailed guidance, you can refer to the Ultralytics documentation on classification datasets.
casperking666
for classification task, do I still need the data.yaml file? If not, how should I carry out validation and testing using the folder structure shown above?
Glenn JocherUltralytics(수정됨)@casperking666 hi there!
For classification tasks with Ultralytics YOLO, you do not need a
data.yamlfile. Instead, you should organize your dataset in the specific folder structure as outlined in the documentation. This structure includes separate directories for training (train), testing (test), and optionally validation (val), with each class having its own subdirectory containing the relevant images.Here’s a quick example of how your dataset should be structured:
dataset/ | |-- train/ | |-- class1/ | | |-- image1.jpg | | |-- image2.jpg | | |-- ... | | | |-- class2/ | | |-- image1.jpg | | |-- image2.jpg | | |-- ... | | | |-- ... | |-- test/ | |-- class1/ | | |-- image1.jpg | | |-- image2.jpg | | |-- ... | | | |-- class2/ | | |-- image1.jpg | | |-- image2.jpg | | |-- ... | | | |-- ... | |-- val/ (optional) | |-- class1/ | | |-- image1.jpg | | |-- image2.jpg | | |-- ... | | | |-- class2/ | | |-- image1.jpg | | |-- image2.jpg | | |-- ... | | | |-- ...To carry out validation and testing, simply ensure that your dataset is organized as shown above. The
train,test, andvaldirectories should each contain subdirectories named after the classes, with the images for each class inside these subdirectories.Here’s how you can train and validate your model using the Ultralytics YOLO API:
Python Example
from ultralytics import YOLO # Load a pretrained classification model model = YOLO("yolo26n-cls.pt") # Train the model results = model.train(data="path/to/dataset", epochs=100, imgsz=640) # Validate the model metrics = model.val(data="path/to/dataset")CLI Example
# Train the model yolo classify train data=path/to/dataset model=yolo26n-cls.pt epochs=100 imgsz=640 # Validate the model yolo classify val data=path/to/dataset model=yolo26n-cls.ptFor more detailed information, you can refer to the Ultralytics documentation on classification datasets.
If you encounter any issues, please ensure you are using the latest version of the packages. If the problem persists, providing a minimum reproducible example can greatly help in diagnosing the issue. You can find more details on creating a reproducible example here.
Happy training! 🚀
casperking666
thanks! I used comet ml during training, and it produced plots metrics/accuracy_top1 VS step and metrics/accuracy_top5 VS step. What sets did they use to generate these plots? I couldn't find any label indicate what set that is. I would assume it's the training set, as the accuracy is getting to close to 100%.
second question, it's not clear to me what set this line of code is using metrics = model.val(data="path/to/dataset")? Is it the validation set or the testing set? If it's the validation set, how would I test the test set as they are all organized under the same directory as you previously showed?
Glenn JocherUltralytics(수정됨)Hi @casperking666,
Thank you for your questions! Let's address them one by one:
-
Comet ML Metrics: The plots for
metrics/accuracy_top1 VS stepandmetrics/accuracy_top5 VS stepgenerated by Comet ML typically use the validation set during training. This is because the validation set provides a more accurate measure of the model's performance on unseen data, which helps in monitoring overfitting. If the accuracy is approaching 100%, it might indicate that the model is overfitting to the training data. To confirm, you can check the Comet ML experiment logs or the training script to see which dataset was used for these metrics. -
Validation and Testing Sets: The
metrics = model.val(data="path/to/dataset")line of code uses the validation set if it exists. If no validation set is provided, it defaults to using the test set. To explicitly validate or test your model, ensure your dataset is organized as follows:dataset/ ├── train/ │ ├── class1/ │ ├── class2/ │ └── ... ├── test/ │ ├── class1/ │ ├── class2/ │ └── ... └── val/ (optional) ├── class1/ ├── class2/ └── ...If you want to validate using the test set, you can specify the
splitargument in thevalmethod:metrics = model.val(data="path/to/dataset", split='test')This ensures that the model uses the test set for validation.
If you encounter any issues, please ensure you are using the latest version of the Ultralytics packages. Providing a minimum reproducible example can greatly help in diagnosing the issue. You can find more details on creating a reproducible example here.
Feel free to reach out if you have any more questions. Happy training! 🚀
-
casperking666
I see, thank you! I have tried to train from scratch using the yaml file like the following:
model = YOLO("yolov8n-cls.yaml")but I got the following error
Traceback (most recent call last): File "/user/work/yf20630/cow-dataset-project/train.py", line 27, in <module> model.save('./models/yolov8s-cow-crops-2080-1e.pt') File "/user/work/yf20630/miniconda3/envs/cow_project/lib/python3.10/site-packages/ultralytics/engine/model.py", line 335, in save torch.save({**self.ckpt, **updates}, filename, use_dill=use_dill) TypeError: 'NoneType' object is not a mappingon this line
model.save('./models/yolov8s-cow-crops-2080-1e.pt')the saving works perfectly fine when using the pretrained model, but wouldn't work here, any thoughts on why?
Glenn JocherUltralytics(수정됨)Hi @casperking666,
Thank you for reaching out and providing the details of your issue!
It looks like you're encountering an error when trying to save a model that was initialized from a YAML configuration file. The error message
TypeError: 'NoneType' object is not a mappingsuggests that there might be an issue with the model's checkpoint data.Here are a few steps to help resolve this:
-
Ensure Proper Initialization: When initializing a model from a YAML file, make sure that the YAML file is correctly configured and that the model is properly built before attempting to save it. Here’s an example of how to initialize and train the model:
from ultralytics import YOLO # Initialize a model from a YAML file model = YOLO("yolo26n-cls.yaml") # Train the model results = model.train(data="path/to/dataset", epochs=100, imgsz=640) # Save the trained model model.save('./models/yolov8s-cow-crops-2080-1e.pt') -
Verify Training Completion: Ensure that the training process completes successfully before saving the model. If the training is interrupted or fails, the model's checkpoint data might not be properly populated.
-
Check for Updates: Verify that you are using the latest version of the Ultralytics package. Sometimes, issues are resolved in newer releases. You can update the package using:
pip install --upgrade ultralytics -
Provide a Reproducible Example: If the issue persists, it would be helpful to provide a minimum reproducible example. This will allow us to better understand the context and provide more accurate assistance. You can find guidelines for creating a reproducible example here.
By following these steps, you should be able to identify and resolve the issue. If you continue to experience problems, please share more details or a reproducible example, and we’ll be happy to assist further.
Best of luck with your training! 🚀
Feel free to reach out if you have any more questions.
-
MrLi-creator
This is my own small_data.yaml file: path: C:/Users/AIX1WX/Documents/10-POC/train_yolov8/yolov8-all-in-one-main_classification/datasets train: train/ val: val/ test: test/
nc: 7 names: ['ABS','Airbag','Background','EPB','EPBGREEN','EPBRed','QMtt']
class_weights: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 3.0]
And my train coda here: from ultralytics import YOLO import os os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE" if name == 'main': model = YOLO("yolov8n-cls.pt") # load a pretrained model (recommended for training) results = model.train(data="small_data.yaml", epochs=30, imgsz=50, device=None, batch=16)
Why will report the error: Dataset not found , missing path C:\Users\AIX1WX\Documents\10-POC\train_yolov8\yolov8-all-in-one-main_detection\datasets\small_data.yaml, attempting download... Downloading https://github.com/ultralytics/yolov5/releases/download/v1.0/small_data.yaml.zip to 'C:\Users\AIX1WX\Documents\10-POC\train_yolov8\yolov8-all-in-one-main_detection\datasets\small_data.yaml.zip'...
Glenn JocherUltralytics@MrLi-creator it looks like the error is due to the dataset path not being correctly recognized. Ensure that the path in your
small_data.yamlfile is correct and accessible. The path should be absolute or relative to your current working directory. Also, make sure the dataset structure follows the required format withtrain,val, andtestdirectories. If the issue persists, please verify it with the latest version of the Ultralytics package. If you need further assistance, feel free to ask!
mopa91
Hello, I have trained a classification model with my custom dataset. Now, I want to validate the model on another dataset. How can I use this: validation_results = model.val(), to validate the model's predictions and get the confusion matrix for the predicted data?
Paula DerrengerUltralyticsHello,
Great to hear that you've trained a classification model with your custom dataset! To validate your model on another dataset and obtain a confusion matrix, you can use the
model.val()method. Here's a concise example:from ultralytics import YOLO # Load your trained model model = YOLO("path/to/your/model.pt") # Validate the model on a new dataset validation_results = model.val(data="path/to/validation/dataset") # Access the confusion matrix confusion_matrix = validation_results.confusion_matrix print(confusion_matrix)Ensure your validation dataset follows the required directory structure. For more details on dataset organization and validation, you can refer to the Image Classification Datasets Overview.
Happy validating! 🚀
momint2020
Hello,
Thank you very much for your reply. It was a great help. Could you please tell me how I can get the F1 score curve for my classification model on the custom dataset?
Paula DerrengerUltralytics@momint2020 hello,
I'm glad to hear the previous information was helpful! To generate an F1 score curve for your classification model on a custom dataset, you can use the
val()method to perform validation and then analyze the predictions to calculate F1 scores at various thresholds.Here’s a basic outline of how you might approach this:
-
Validate the Model: First, run validation to get the predictions.
from ultralytics import YOLO # Load your trained model model = YOLO("path/to/your/model.pt") # Validate the model on your custom dataset validation_results = model.val(data="path/to/your/validation/dataset") -
Calculate F1 Scores: You'll need to write a custom function or script to calculate the F1 score for each class at different thresholds, using the predictions obtained. This involves calculating precision and recall for each threshold and then using these to calculate the F1 score.
-
Plot the Curve: Use a plotting library like
matplotlibto plot the F1 scores against the thresholds.
Unfortunately, direct support for plotting F1 curves isn't built into the Ultralytics framework, so this will require some custom implementation on your part.
If you need further guidance on the specifics of calculating and plotting the F1 score, please let me know!
-