์ฝ˜ํ…์ธ ๋กœ ๊ฑด๋„ˆ๋›ฐ๊ธฐ

Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ

Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ๋Š” 3D ์ถ”์ , ๋™์ž‘ ์˜ˆ์ธก, ์Šคํ…Œ๋ ˆ์˜ค ๊นŠ์ด ์ถ”์ •๊ณผ ๊ฐ™์€ ์ž์œจ ์ฃผํ–‰ ์ž‘์—…์˜ ์—ฐ๊ตฌ๋ฅผ ์ง€์›ํ•˜๊ธฐ ์œ„ํ•ด ์„ค๊ณ„๋œ ๋ฐ์ดํ„ฐ ๋ชจ์Œ์ž…๋‹ˆ๋‹ค. Argo AI์—์„œ ๊ฐœ๋ฐœํ•œ ์ด ๋ฐ์ดํ„ฐ ์„ธํŠธ๋Š” ๊ณ ํ•ด์ƒ๋„ ์ด๋ฏธ์ง€, LiDAR ํฌ์ธํŠธ ํด๋ผ์šฐ๋“œ, ์ง€๋„ ๋ฐ์ดํ„ฐ๋ฅผ ํฌํ•จํ•œ ๋‹ค์–‘ํ•œ ๊ณ ํ’ˆ์งˆ ์„ผ์„œ ๋ฐ์ดํ„ฐ๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.

์ฐธ๊ณ 

Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ *.zip ํ›ˆ๋ จ์— ํ•„์š”ํ•œ ํŒŒ์ผ์€ Ford์˜ Argo AI ์ข…๋ฃŒ ์ดํ›„ Amazon S3์—์„œ ์ œ๊ฑฐ๋˜์—ˆ์ง€๋งŒ, ๋‹ค์Œ์—์„œ ์ˆ˜๋™์œผ๋กœ ๋‹ค์šด๋กœ๋“œํ•  ์ˆ˜ ์žˆ๋„๋ก ํ–ˆ์Šต๋‹ˆ๋‹ค. Google ๋“œ๋ผ์ด๋ธŒ.

์ฃผ์š” ๊ธฐ๋Šฅ

  • Argoverse์—๋Š” 1,263๊ฐœ์˜ ๊ฐœ๋ณ„ ์žฅ๋ฉด์— ๊ฑธ์ณ 290๋งŒ ๊ฐœ ์ด์ƒ์˜ ๋ผ๋ฒจ์ด ์ง€์ •๋œ 3D ์˜ค๋ธŒ์ ํŠธ ํŠธ๋ž™๊ณผ 500๋งŒ ๊ฐœ์˜ ์˜ค๋ธŒ์ ํŠธ ์ธ์Šคํ„ด์Šค๊ฐ€ ํฌํ•จ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค.
  • ์ด ๋ฐ์ดํ„ฐ ์„ธํŠธ์—๋Š” ๊ณ ํ•ด์ƒ๋„ ์นด๋ฉ”๋ผ ์ด๋ฏธ์ง€, LiDAR ํฌ์ธํŠธ ํด๋ผ์šฐ๋“œ, ํ’๋ถ€ํ•œ ์ฃผ์„์ด ๋‹ฌ๋ฆฐ HD ์ง€๋„๊ฐ€ ํฌํ•จ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค.
  • ์ฃผ์„์—๋Š” ๊ฐœ์ฒด์— ๋Œ€ํ•œ 3D ๊ฒฝ๊ณ„ ์ƒ์ž, ๊ฐœ์ฒด ํŠธ๋ž™ ๋ฐ ๊ถค์  ์ •๋ณด๊ฐ€ ํฌํ•จ๋ฉ๋‹ˆ๋‹ค.
  • Argoverse๋Š” 3D ์ถ”์ , ๋ชจ์…˜ ์˜ˆ์ธก, ์Šคํ…Œ๋ ˆ์˜ค ๊นŠ์ด ์ถ”์ • ๋“ฑ ๋‹ค์–‘ํ•œ ์ž‘์—…์„ ์œ„ํ•œ ์—ฌ๋Ÿฌ ํ•˜์œ„ ์ง‘ํ•ฉ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.

๋ฐ์ดํ„ฐ ์„ธํŠธ ๊ตฌ์กฐ

Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ๋Š” ์„ธ ๊ฐ€์ง€ ์ฃผ์š” ํ•˜์œ„ ์ง‘ํ•ฉ์œผ๋กœ ๊ตฌ์„ฑ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค:

  1. ์•„๋ฅด๊ณ ๋ฒ„์Šค 3D ํŠธ๋ž˜ํ‚น: ์ด ํ•˜์œ„ ์ง‘ํ•ฉ์—๋Š” 3D ๊ฐ์ฒด ์ถ”์  ์ž‘์—…์— ์ค‘์ ์„ ๋‘” 290K ์ด์ƒ์˜ ๋ ˆ์ด๋ธ”์ด ์ง€์ •๋œ 3D ๊ฐ์ฒด ํŠธ๋ž™์ด ์žˆ๋Š” 113๊ฐœ์˜ ์žฅ๋ฉด์ด ํฌํ•จ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์—๋Š” LiDAR ํฌ์ธํŠธ ํด๋ผ์šฐ๋“œ, ์นด๋ฉ”๋ผ ์ด๋ฏธ์ง€, ์„ผ์„œ ๋ณด์ • ์ •๋ณด๊ฐ€ ํฌํ•จ๋ฉ๋‹ˆ๋‹ค.
  2. ์•„๋ฅด๊ณ ๋ฒ„์Šค ๋ชจ์…˜ ์˜ˆ์ธก: ์ด ํ•˜์œ„ ์ง‘ํ•ฉ์€ 60์‹œ๊ฐ„์˜ ์ฃผํ–‰ ๋ฐ์ดํ„ฐ์—์„œ ์ˆ˜์ง‘ํ•œ 32๋งŒ 4์ฒœ ๊ฐœ์˜ ์ฐจ๋Ÿ‰ ๊ถค์ ์œผ๋กœ ๊ตฌ์„ฑ๋˜๋ฉฐ, ๋ชจ์…˜ ์˜ˆ์ธก ์ž‘์—…์— ์ ํ•ฉํ•ฉ๋‹ˆ๋‹ค.
  3. ์•„๊ณ ๋ฒ„์Šค ์Šคํ…Œ๋ ˆ์˜ค ๊นŠ์ด ์ถ”์ •: ์ด ํ•˜์œ„ ์ง‘ํ•ฉ์€ ์Šคํ…Œ๋ ˆ์˜ค ๊นŠ์ด ์ถ”์ • ์ž‘์—…์„ ์œ„ํ•ด ์„ค๊ณ„๋˜์—ˆ์œผ๋ฉฐ, ์‹ค์ธก ๊นŠ์ด ์ถ”์ •์„ ์œ„ํ•œ 10K ์ด์ƒ์˜ ์Šคํ…Œ๋ ˆ์˜ค ์ด๋ฏธ์ง€ ์Œ๊ณผ ํ•ด๋‹น LiDAR ํฌ์ธํŠธ ํด๋ผ์šฐ๋“œ๊ฐ€ ํฌํ•จ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค.

์• ํ”Œ๋ฆฌ์ผ€์ด์…˜

Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ๋Š” 3D ๋ฌผ์ฒด ์ถ”์ , ๋™์ž‘ ์˜ˆ์ธก, ์Šคํ…Œ๋ ˆ์˜ค ๊นŠ์ด ์ถ”์ •๊ณผ ๊ฐ™์€ ์ž์œจ ์ฃผํ–‰ ์ž‘์—…์—์„œ ๋”ฅ ๋Ÿฌ๋‹ ๋ชจ๋ธ์„ ํ›ˆ๋ จํ•˜๊ณ  ํ‰๊ฐ€ํ•˜๋Š” ๋ฐ ๋„๋ฆฌ ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค. ๋ฐ์ดํ„ฐ ์„ธํŠธ์˜ ๋‹ค์–‘ํ•œ ์„ผ์„œ ๋ฐ์ดํ„ฐ, ๊ฐ์ฒด ์ฃผ์„, ์ง€๋„ ์ •๋ณด๋Š” ์ž์œจ ์ฃผํ–‰ ๋ถ„์•ผ์˜ ์—ฐ๊ตฌ์ž์™€ ์‹ค๋ฌด์ž์—๊ฒŒ ์œ ์šฉํ•œ ๋ฆฌ์†Œ์Šค์ž…๋‹ˆ๋‹ค.

๋ฐ์ดํ„ฐ ์„ธํŠธ YAML

๋ฐ์ดํ„ฐ ์„ธํŠธ ๊ตฌ์„ฑ์„ ์ •์˜ํ•˜๋Š” ๋ฐ๋Š” YAML(๋˜ ๋‹ค๋ฅธ ๋งˆํฌ์—… ์–ธ์–ด) ํŒŒ์ผ์ด ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์—๋Š” ๋ฐ์ดํ„ฐ ์„ธํŠธ์˜ ๊ฒฝ๋กœ, ํด๋ž˜์Šค ๋ฐ ๊ธฐํƒ€ ๊ด€๋ จ ์ •๋ณด์— ๋Œ€ํ•œ ์ •๋ณด๊ฐ€ ํฌํ•จ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ์˜ ๊ฒฝ์šฐ, ๋ฐ์ดํ„ฐ ์„ธํŠธ์˜ Argoverse.yaml ํŒŒ์ผ์€ ๋‹ค์Œ ์œ„์น˜์—์„œ ์œ ์ง€๋ฉ๋‹ˆ๋‹ค. https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/Argoverse.yaml.

ultralytics/cfg/datasets/Argoverse.yaml

# Ultralytics YOLO ๐Ÿš€, AGPL-3.0 license
# Argoverse-HD dataset (ring-front-center camera) https://www.cs.cmu.edu/~mengtial/proj/streaming/ by Argo AI
# Documentation: https://docs.ultralytics.com/datasets/detect/argoverse/
# Example usage: yolo train data=Argoverse.yaml
# parent
# โ”œโ”€โ”€ ultralytics
# โ””โ”€โ”€ datasets
#     โ””โ”€โ”€ Argoverse  โ† downloads here (31.5 GB)

# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: ../datasets/Argoverse # dataset root dir
train: Argoverse-1.1/images/train/ # train images (relative to 'path') 39384 images
val: Argoverse-1.1/images/val/ # val images (relative to 'path') 15062 images
test: Argoverse-1.1/images/test/ # test images (optional) https://eval.ai/web/challenges/challenge-page/800/overview

# Classes
names:
  0: person
  1: bicycle
  2: car
  3: motorcycle
  4: bus
  5: truck
  6: traffic_light
  7: stop_sign

# Download script/URL (optional) ---------------------------------------------------------------------------------------
download: |
  import json
  from tqdm import tqdm
  from ultralytics.utils.downloads import download
  from pathlib import Path

  def argoverse2yolo(set):
      labels = {}
      a = json.load(open(set, "rb"))
      for annot in tqdm(a['annotations'], desc=f"Converting {set} to YOLOv5 format..."):
          img_id = annot['image_id']
          img_name = a['images'][img_id]['name']
          img_label_name = f'{img_name[:-3]}txt'

          cls = annot['category_id']  # instance class id
          x_center, y_center, width, height = annot['bbox']
          x_center = (x_center + width / 2) / 1920.0  # offset and scale
          y_center = (y_center + height / 2) / 1200.0  # offset and scale
          width /= 1920.0  # scale
          height /= 1200.0  # scale

          img_dir = set.parents[2] / 'Argoverse-1.1' / 'labels' / a['seq_dirs'][a['images'][annot['image_id']]['sid']]
          if not img_dir.exists():
              img_dir.mkdir(parents=True, exist_ok=True)

          k = str(img_dir / img_label_name)
          if k not in labels:
              labels[k] = []
          labels[k].append(f"{cls} {x_center} {y_center} {width} {height}\n")

      for k in labels:
          with open(k, "w") as f:
              f.writelines(labels[k])


  # Download 'https://argoverse-hd.s3.us-east-2.amazonaws.com/Argoverse-HD-Full.zip' (deprecated S3 link)
  dir = Path(yaml['path'])  # dataset root dir
  urls = ['https://drive.google.com/file/d/1st9qW3BeIwQsnR0t8mRpvbsSWIo16ACi/view?usp=drive_link']
  print("\n\nWARNING: Argoverse dataset MUST be downloaded manually, autodownload will NOT work.")
  print(f"WARNING: Manually download Argoverse dataset '{urls[0]}' to '{dir}' and re-run your command.\n\n")
  # download(urls, dir=dir)

  # Convert
  annotations_dir = 'Argoverse-HD/annotations/'
  (dir / 'Argoverse-1.1' / 'tracking').rename(dir / 'Argoverse-1.1' / 'images')  # rename 'tracking' to 'images'
  for d in "train.json", "val.json":
      argoverse2yolo(dir / annotations_dir / d)  # convert Argoverse annotations to YOLO labels

์‚ฌ์šฉ๋ฒ•

์ด๋ฏธ์ง€ ํฌ๊ธฐ๊ฐ€ 640์ธ 100๊ฐœ์˜ ์—ํฌํฌ์— ๋Œ€ํ•œ Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ์—์„œ YOLOv8n ๋ชจ๋ธ์„ ํ›ˆ๋ จํ•˜๋ ค๋ฉด ๋‹ค์Œ ์ฝ”๋“œ ์กฐ๊ฐ์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ์ธ์ˆ˜์˜ ์ „์ฒด ๋ชฉ๋ก์€ ๋ชจ๋ธ ํ•™์Šต ํŽ˜์ด์ง€๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

์—ด์ฐจ ์˜ˆ์‹œ

from ultralytics import YOLO

# Load a model
model = YOLO('yolov8n.pt')  # load a pretrained model (recommended for training)

# Train the model
results = model.train(data='Argoverse.yaml', epochs=100, imgsz=640)
# Start training from a pretrained *.pt model
yolo detect train data=Argoverse.yaml model=yolov8n.pt epochs=100 imgsz=640

์ƒ˜ํ”Œ ๋ฐ์ดํ„ฐ ๋ฐ ์ฃผ์„

Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ์—๋Š” ์นด๋ฉ”๋ผ ์ด๋ฏธ์ง€, LiDAR ํฌ์ธํŠธ ํด๋ผ์šฐ๋“œ, HD ์ง€๋„ ์ •๋ณด ๋“ฑ ๋‹ค์–‘ํ•œ ์„ผ์„œ ๋ฐ์ดํ„ฐ๊ฐ€ ํฌํ•จ๋˜์–ด ์žˆ์–ด ์ž์œจ์ฃผํ–‰ ์ž‘์—…์— ๋Œ€ํ•œ ํ’๋ถ€ํ•œ ์ปจํ…์ŠคํŠธ๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. ๋‹ค์Œ์€ ๋ฐ์ดํ„ฐ ์„ธํŠธ์˜ ๋ช‡ ๊ฐ€์ง€ ๋ฐ์ดํ„ฐ ์˜ˆ์‹œ์™€ ํ•ด๋‹น ์ฃผ์„์ž…๋‹ˆ๋‹ค:

๋ฐ์ดํ„ฐ ์„ธํŠธ ์ƒ˜ํ”Œ ์ด๋ฏธ์ง€

  • ์•„๋ฅด๊ณ ๋ฒ„์Šค 3D ํŠธ๋ž˜ํ‚น: ์ด ์ด๋ฏธ์ง€๋Š” 3D ๊ฐ์ฒด์— 3D ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค๋กœ ์ฃผ์„์ด ๋‹ฌ๋ฆฐ 3D ๊ฐ์ฒด ์ถ”์ ์˜ ์˜ˆ๋ฅผ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค. ์ด ๋ฐ์ดํ„ฐ ์„ธํŠธ๋Š” ์ด ์ž‘์—…์„ ์œ„ํ•œ ๋ชจ๋ธ์„ ์‰ฝ๊ฒŒ ๊ฐœ๋ฐœํ•  ์ˆ˜ ์žˆ๋„๋ก LiDAR ํฌ์ธํŠธ ํด๋ผ์šฐ๋“œ์™€ ์นด๋ฉ”๋ผ ์ด๋ฏธ์ง€๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.

์ด ์˜ˆ์‹œ๋Š” Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ์— ํฌํ•จ๋œ ๋ฐ์ดํ„ฐ์˜ ๋‹ค์–‘์„ฑ๊ณผ ๋ณต์žก์„ฑ์„ ๋ณด์—ฌ์ฃผ๋ฉฐ, ์ž์œจ์ฃผํ–‰ ์ž‘์—…์—์„œ ๊ณ ํ’ˆ์งˆ ์„ผ์„œ ๋ฐ์ดํ„ฐ์˜ ์ค‘์š”์„ฑ์„ ๊ฐ•์กฐํ•ฉ๋‹ˆ๋‹ค.

์ธ์šฉ ๋ฐ ๊ฐ์‚ฌ

์—ฐ๊ตฌ ๋˜๋Š” ๊ฐœ๋ฐœ ์ž‘์—…์— Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ ๋‹ค์Œ ๋…ผ๋ฌธ์„ ์ธ์šฉํ•ด ์ฃผ์„ธ์š”:

@inproceedings{chang2019argoverse,
  title={Argoverse: 3D Tracking and Forecasting with Rich Maps},
  author={Chang, Ming-Fang and Lambert, John and Sangkloy, Patsorn and Singh, Jagjeet and Bak, Slawomir and Hartnett, Andrew and Wang, Dequan and Carr, Peter and Lucey, Simon and Ramanan, Deva and others},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={8748--8757},
  year={2019}
}

์ž์œจ ์ฃผํ–‰ ์—ฐ๊ตฌ ์ปค๋ฎค๋‹ˆํ‹ฐ๋ฅผ ์œ„ํ•œ ๊ท€์ค‘ํ•œ ๋ฆฌ์†Œ์Šค์ธ Argoverse ๋ฐ์ดํ„ฐ์„ธํŠธ๋ฅผ ๋งŒ๋“ค๊ณ  ์œ ์ง€ ๊ด€๋ฆฌํ•ด ์ฃผ์‹  Argo AI์— ๊ฐ์‚ฌ์˜ ๋ง์”€์„ ์ „ํ•ฉ๋‹ˆ๋‹ค. Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ์™€ ์ œ์ž‘์ž์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ Argoverse ๋ฐ์ดํ„ฐ ์„ธํŠธ ์›น์‚ฌ์ดํŠธ๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.



์ƒ์„ฑ 2023-11-12, ์—…๋ฐ์ดํŠธ 2024-01-14
์ž‘์„ฑ์ž: glenn-jocher (4), Laughing-q (1)

๋Œ“๊ธ€