์ฝ˜ํ…์ธ ๋กœ ๊ฑด๋„ˆ๋›ฐ๊ธฐ

TensorRT ๋ฐ DeepStream SDK๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ NVIDIA Jetson์— ๋ฐฐํฌํ•ฉ๋‹ˆ๋‹ค.

์ด ๊ฐ€์ด๋“œ์—์„œ๋Š” ํ•™์Šต๋œ ๋ชจ๋ธ์„ NVIDIA Jetson ํ”Œ๋žซํผ์— ๋ฐฐํฌํ•˜๊ณ  TensorRT ๋ฐ DeepStream SDK๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ถ”๋ก ์„ ์ˆ˜ํ–‰ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์„ค๋ช…ํ•ฉ๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ๋Š” TensorRT ์„ ์‚ฌ์šฉํ•˜์—ฌ Jetson ํ”Œ๋žซํผ์—์„œ ์ถ”๋ก  ์„ฑ๋Šฅ์„ ๊ทน๋Œ€ํ™”ํ•ฉ๋‹ˆ๋‹ค.

ํ•˜๋“œ์›จ์–ด ๊ฒ€์ฆ

์ด ๊ฐ€์ด๋“œ๋Š” ๋‹ค์Œ Jetson ์žฅ์น˜์—์„œ ํ…Œ์ŠคํŠธ ๋ฐ ๊ฒ€์ฆ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

์‹œ์ž‘ํ•˜๊ธฐ ์ „์—

์—ฌ๊ธฐ์—๋Š” ์ด ๊ฐ€์ด๋“œ์— ํ•„์š”ํ•œ CUDA, TensorRT ๋ฐ DeepStream SDK๊ฐ€ ํฌํ•จ๋˜์–ด ์žˆ์œผ๋ฏ€๋กœ Jetson ์žฅ์น˜์— ๋ชจ๋“  SDK ๊ตฌ์„ฑ ์š”์†Œ ๋ฐ DeepStream SDK์™€ ํ•จ๊ป˜ JetPack SDK๋ฅผ ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ์„ค์น˜ํ–ˆ๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.

JetPack SDK๋Š” ํ•˜๋“œ์›จ์–ด ๊ฐ€์† AI ์—ฃ์ง€ ๊ฐœ๋ฐœ์„ ์œ„ํ•œ ์™„๋ฒฝํ•œ ๊ฐœ๋ฐœ ํ™˜๊ฒฝ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. ๋ชจ๋“  Jetson ๋ชจ๋“ˆ ๋ฐ ๊ฐœ๋ฐœ์ž ํ‚คํŠธ๋Š” JetPack SDK์—์„œ ์ง€์›๋ฉ๋‹ˆ๋‹ค.

์„ค์น˜ ๋ฐฉ๋ฒ•์—๋Š” ํฌ๊ฒŒ ๋‘ ๊ฐ€์ง€๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค,

  1. SD ์นด๋“œ ์ด๋ฏธ์ง€ ๋ฐฉ๋ฒ•
  2. NVIDIA SDK ๊ด€๋ฆฌ์ž ๋ฐฉ๋ฒ•

NVIDIA ๊ณต์‹ ์›น์‚ฌ์ดํŠธ์—์„œ ๋งค์šฐ ์ž์„ธํ•œ ์„ค์น˜ ๊ฐ€์ด๋“œ๋ฅผ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์œ„์—์„œ ์–ธ๊ธ‰ํ•œ ์žฌ์ปดํ“จํ„ฐ J1010 ๋ฐ ์žฌ์ปดํ“จํ„ฐ J2021์— ํ•ด๋‹นํ•˜๋Š” ๊ฐ€์ด๋“œ๋„ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

ํ•„์š”ํ•œ ํŒจํ‚ค์ง€ ์„ค์น˜

  • 1๋‹จ๊ณ„. Jetson ์žฅ์น˜์˜ ํ„ฐ๋ฏธ๋„์— ์•ก์„ธ์Šคํ•˜์—ฌ pip๋ฅผ ์„ค์น˜ํ•˜๊ณ  ์—…๊ทธ๋ ˆ์ด๋“œํ•ฉ๋‹ˆ๋‹ค.
sudo apt update
sudo apt install -y python3-pip
pip3 install --upgrade pip
  • 2๋‹จ๊ณ„. ๋‹ค์Œ ๋ฆฌํฌ์ง€ํ† ๋ฆฌ๋ฅผ ๋ณต์ œํ•ฉ๋‹ˆ๋‹ค.
git clone https://github.com/ultralytics/yolov5
  • 3๋‹จ๊ณ„ ์š”๊ตฌ ์‚ฌํ•ญ.txt ์—ด๊ธฐ
cd yolov5
vi requirements.txt
  • 5๋‹จ๊ณ„. ๋‹ค์Œ ์ค„์„ ํŽธ์ง‘ํ•ฉ๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ ๋จผ์ € i๋ฅผ ๋ˆŒ๋Ÿฌ ํŽธ์ง‘ ๋ชจ๋“œ๋กœ ๋“ค์–ด๊ฐ€์•ผ ํ•ฉ๋‹ˆ๋‹ค. ESC๋ฅผ ๋ˆ„๋ฅธ ๋‹ค์Œ :wq๋ฅผ ์ž…๋ ฅํ•˜์—ฌ ์ €์žฅํ•˜๊ณ  ์ข…๋ฃŒํ•ฉ๋‹ˆ๋‹ค.
# torch>=1.8.0
# torchvision>=0.9.0

์ฐธ๊ณ : torch ๋ฐ torchvision์€ ๋‚˜์ค‘์— ์„ค์น˜๋  ์˜ˆ์ •์ด๋ฏ€๋กœ ์ง€๊ธˆ์€ ์ œ์™ธ๋ฉ๋‹ˆ๋‹ค.

  • 6๋‹จ๊ณ„. ์•„๋ž˜ ์ข…์†์„ฑ ์„ค์น˜
sudo apt install -y libfreetype6-dev
  • 7๋‹จ๊ณ„. ํ•„์š”ํ•œ ํŒจํ‚ค์ง€ ์„ค์น˜
pip3 install -r requirements.txt

PyTorch ๋ฐ ํ† ์น˜๋น„์ „ ์„ค์น˜

PyTorch ๋ฐ Torchvision์€ ARM aarch64 ์•„ํ‚คํ…์ฒ˜ ๊ธฐ๋ฐ˜์˜ Jetson ํ”Œ๋žซํผ์—์„œ ์‹คํ–‰ํ•˜๊ธฐ ์œ„ํ•ด ํ˜ธํ™˜๋˜์ง€ ์•Š์œผ๋ฏ€๋กœ pip์—์„œ ์„ค์น˜ํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค. ๋”ฐ๋ผ์„œ ๋ฏธ๋ฆฌ ๋นŒ๋“œ๋œ PyTorch pip ํœ ์„ ์ˆ˜๋™์œผ๋กœ ์„ค์น˜ํ•˜๊ณ  ์†Œ์Šค์—์„œ Torchvision์„ ์ปดํŒŒ์ผ/์„ค์น˜ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

์ด ํŽ˜์ด์ง€๋ฅผ ๋ฐฉ๋ฌธํ•˜์—ฌ PyTorch ๋ฐ ํ† ์น˜๋น„์ „ ๋งํฌ์— ๋ชจ๋‘ ์•ก์„ธ์Šคํ•˜์„ธ์š”.

JetPack 4.6 ์ด์ƒ์—์„œ ์ง€์›๋˜๋Š” ๋ฒ„์ „์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

PyTorch v1.10.0

JetPack 4.4(L4T R32.4.3) / JetPack 4.4.1(L4T R32.4.4) / JetPack 4.5(L4T R32.5.0) / JetPack 4.5.1(L4T R32.5.1)์—์„œ ์ง€์› Python 3.6

PyTorch v1.12.0

์ ฏํŒฉ 5.0(L4T R34.1.0)/์ ฏํŒฉ 5.0.1(L4T R34.1.1)/์ ฏํŒฉ 5.0.2(L4T R35.1.0)์—์„œ ์ง€์› Python 3.8

wget <URL> -O <file_name>
pip3 install <file_name>

์˜ˆ๋ฅผ ๋“ค์–ด, ์—ฌ๊ธฐ์„œ๋Š” JP4.6.1์„ ์‹คํ–‰ํ•˜๊ณ  ์žˆ์œผ๋ฏ€๋กœ PyTorch v1.10.0์„ ์„ ํƒํ•ฉ๋‹ˆ๋‹ค.

cd ~
sudo apt-get install -y libopenblas-base libopenmpi-dev
wget https://nvidia.box.com/shared/static/fjtbno0vpo676a25cgvuqc1wty0fkkg6.whl -O torch-1.10.0-cp36-cp36m-linux_aarch64.whl
pip3 install torch-1.10.0-cp36-cp36m-linux_aarch64.whl
  • 2๋‹จ๊ณ„. ์„ค์น˜ํ•œ PyTorch ๋ฒ„์ „์— ๋”ฐ๋ผ ํ† ์น˜๋น„์ „์„ ์„ค์น˜ํ•ฉ๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค์–ด PyTorch v1.10.0์„ ์„ ํƒํ–ˆ์œผ๋ฏ€๋กœ Torchvision v0.11.1์„ ์„ ํƒํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
sudo apt install -y libjpeg-dev zlib1g-dev
git clone --branch v0.11.1 https://github.com/pytorch/vision torchvision
cd torchvision
sudo python3 setup.py install

๋‹ค์Œ์€ PyTorch ๋ฒ„์ „์— ๋”ฐ๋ผ ์„ค์น˜ํ•ด์•ผ ํ•˜๋Š” ํ•ด๋‹น ํ† ์น˜๋น„์ „ ๋ฒ„์ „ ๋ชฉ๋ก์ž…๋‹ˆ๋‹ค:

  • PyTorch v1.10 - ํ† ์น˜๋น„์ „ V0.11.1
  • PyTorch v1.12 - ํ† ์น˜๋น„์ „ V0.13.0

๋”ฅ์ŠคํŠธ๋ฆผ ๊ตฌ์„ฑ YOLOv5

  • 1๋‹จ๊ณ„. ๋‹ค์Œ ๋ฆฌํฌ์ง€ํ† ๋ฆฌ๋ฅผ ๋ณต์ œํ•ฉ๋‹ˆ๋‹ค.
cd ~
git clone https://github.com/marcoslucianops/DeepStream-Yolo
  • 2๋‹จ๊ณ„. DeepStream-Yolo /utils์—์„œ gen_wts_yoloV5.py๋ฅผ ๋‹ค์Œ ๋””๋ ‰ํ† ๋ฆฌ์— ๋ณต์‚ฌํ•ฉ๋‹ˆ๋‹ค. yolov5 ๋””๋ ‰ํ† ๋ฆฌ
cp DeepStream-Yolo/utils/gen_wts_yoloV5.py yolov5
  • 3๋‹จ๊ณ„. yolov5 ๋ฆฌํฌ์ง€ํ† ๋ฆฌ ๋‚ด์—์„œ YOLOv5 ๋ฆด๋ฆฌ์Šค์—์„œ pt ํŒŒ์ผ์„ ๋‹ค์šด๋กœ๋“œํ•ฉ๋‹ˆ๋‹ค(์˜ˆ: YOLOv5s 6.1).
cd yolov5
wget https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5s.pt
  • 4๋‹จ๊ณ„. cfg ๋ฐ wts ํŒŒ์ผ ์ƒ์„ฑํ•˜๊ธฐ
python3 gen_wts_yoloV5.py -w yolov5s.pt

์ฐธ๊ณ : ์ถ”๋ก  ํฌ๊ธฐ ๋ณ€๊ฒฝํ•˜๊ธฐ(๊ธฐ๋ณธ๊ฐ’: 640)

-s SIZE
--size SIZE
-s HEIGHT WIDTH
--size HEIGHT WIDTH

Example for 1280:

-s 1280
or
-s 1280 1280
  • 5๋‹จ๊ณ„. ์ƒ์„ฑ๋œ cfg ๋ฐ wts ํŒŒ์ผ์„ DeepStream-Yolo ํด๋”์— ๋ณต์‚ฌํ•ฉ๋‹ˆ๋‹ค.
cp yolov5s.cfg ~/DeepStream-Yolo
cp yolov5s.wts ~/DeepStream-Yolo
  • 6๋‹จ๊ณ„. DeepStream-Yolo ํด๋”๋ฅผ ์—ด๊ณ  ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ์ปดํŒŒ์ผํ•ฉ๋‹ˆ๋‹ค.
cd ~/DeepStream-Yolo
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo  # for DeepStream 6.1
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo  # for DeepStream 6.0.1 / 6.0
  • 7๋‹จ๊ณ„. ๋ชจ๋ธ์— ๋”ฐ๋ผ config_infer_primary_yoloV5.txt ํŒŒ์ผ์„ ํŽธ์ง‘ํ•ฉ๋‹ˆ๋‹ค.
[property]
...
custom-network-config=yolov5s.cfg
model-file=yolov5s.wts
...
  • 8๋‹จ๊ณ„. deepstream_app_config ํŒŒ์ผ์„ ํŽธ์ง‘ํ•ฉ๋‹ˆ๋‹ค.
...
[primary-gie]
...
config-file=config_infer_primary_yoloV5.txt
  • 9๋‹จ๊ณ„. deepstream_app_config ํŒŒ์ผ์—์„œ ๋น„๋””์˜ค ์†Œ์Šค๋ฅผ ๋ณ€๊ฒฝํ•ฉ๋‹ˆ๋‹ค. ์•„๋ž˜์—์„œ ๋ณผ ์ˆ˜ ์žˆ๋“ฏ์ด ๊ธฐ๋ณธ ๋น„๋””์˜ค ํŒŒ์ผ์ด ๋กœ๋“œ๋ฉ๋‹ˆ๋‹ค.
...
[source0]
...
uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4

์ถ”๋ก  ์‹คํ–‰

deepstream-app -c deepstream_app_config.txt
YOLOv5 ๋”ฅ์ŠคํŠธ๋ฆผ FP32

์œ„์˜ ๊ฒฐ๊ณผ๋Š” FP32 ๋ฐ YOLOv5s 640x640์˜ Jetson Xavier NX์—์„œ ์‹คํ–‰ํ•œ ๊ฒฐ๊ณผ์ž…๋‹ˆ๋‹ค. FPS๊ฐ€ 30 ์ •๋„์ž„์„ ์•Œ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

INT8 ๋ณด์ •

์ถ”๋ก ์— INT8 ์ •๋ฐ€๋„๋ฅผ ์‚ฌ์šฉํ•˜๋ ค๋ฉด ๋‹ค์Œ ๋‹จ๊ณ„๋ฅผ ๋”ฐ๋ผ์•ผ ํ•ฉ๋‹ˆ๋‹ค.

  • 1๋‹จ๊ณ„. OpenCV ์„ค์น˜
sudo apt-get install libopencv-dev
  • 2๋‹จ๊ณ„. OpenCV๋ฅผ ์ง€์›ํ•˜๋Š” nvdsinfer_custom_impl_Yolo ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ์ปดํŒŒ์ผ/๋‹ค์‹œ ์ปดํŒŒ์ผํ•ฉ๋‹ˆ๋‹ค.
cd ~/DeepStream-Yolo
CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo  # for DeepStream 6.1
CUDA_VER=10.2 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo  # for DeepStream 6.0.1 / 6.0
  • 3๋‹จ๊ณ„. COCO ๋ฐ์ดํ„ฐ ์„ธํŠธ์˜ ๊ฒฝ์šฐ, val2017์„ ๋‹ค์šด๋กœ๋“œํ•˜๊ณ  ์••์ถ•์„ ํ‘ผ ํ›„ DeepStream-Yolo ํด๋”๋กœ ์ด๋™ํ•ฉ๋‹ˆ๋‹ค.

  • 4๋‹จ๊ณ„. ๋ณด์ • ์ด๋ฏธ์ง€๋ฅผ ์œ„ํ•œ ์ƒˆ ๋””๋ ‰ํ„ฐ๋ฆฌ ๋งŒ๋“ค๊ธฐ

mkdir calibration
  • 5๋‹จ๊ณ„. ๋‹ค์Œ์„ ์‹คํ–‰ํ•˜์—ฌ COCO ๋ฐ์ดํ„ฐ ์„ธํŠธ์—์„œ 1000๊ฐœ์˜ ๋ฌด์ž‘์œ„ ์ด๋ฏธ์ง€๋ฅผ ์„ ํƒํ•˜์—ฌ ๋ณด์ •์„ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.
for jpg in $(ls -1 val2017/*.jpg | sort -R | head -1000); do \
    cp ${jpg} calibration/; \
done

์ฐธ๊ณ : NVIDIA๋Š” ์ข‹์€ ์ •ํ™•๋„๋ฅผ ์–ป์œผ๋ ค๋ฉด ์ตœ์†Œ 500๊ฐœ์˜ ์ด๋ฏธ์ง€๋ฅผ ๊ถŒ์žฅํ•ฉ๋‹ˆ๋‹ค. ์ด ์˜ˆ์ œ์—์„œ๋Š” ๋” ๋‚˜์€ ์ •ํ™•๋„๋ฅผ ์–ป๊ธฐ ์œ„ํ•ด 1000๊ฐœ์˜ ์ด๋ฏธ์ง€๊ฐ€ ์„ ํƒ๋˜์—ˆ์Šต๋‹ˆ๋‹ค(์ด๋ฏธ์ง€๊ฐ€ ๋งŽ์„์ˆ˜๋ก ์ •ํ™•๋„๊ฐ€ ๋†’์•„์ง). INT8_CALIB_BATCH_SIZE ๊ฐ’์ด ๋†’์„์ˆ˜๋ก ์ •ํ™•๋„๊ฐ€ ๋†’์•„์ง€๊ณ  ๋ณด์ • ์†๋„๊ฐ€ ๋นจ๋ผ์ง‘๋‹ˆ๋‹ค. GPU ๋ฉ”๋ชจ๋ฆฌ์— ๋งž๊ฒŒ ์„ค์ •ํ•˜์„ธ์š”. ํ—ค๋“œ -1000๋ถ€ํ„ฐ ์„ค์ •ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค์–ด ์ด๋ฏธ์ง€๊ฐ€ 2000๊ฐœ์ธ ๊ฒฝ์šฐ -2000์„ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ด ๊ณผ์ •์€ ์‹œ๊ฐ„์ด ์˜ค๋ž˜ ๊ฑธ๋ฆด ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

  • 6๋‹จ๊ณ„. ์„ ํƒํ•œ ๋ชจ๋“  ์ด๋ฏธ์ง€๊ฐ€ ํฌํ•จ๋œ calibration.txt ํŒŒ์ผ์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค.
realpath calibration/*jpg > calibration.txt
  • 7๋‹จ๊ณ„. ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •
export INT8_CALIB_IMG_PATH=calibration.txt
export INT8_CALIB_BATCH_SIZE=1
  • 8๋‹จ๊ณ„. config_infer_primary_yoloV5.txt ํŒŒ์ผ์„ ์—…๋ฐ์ดํŠธํ•ฉ๋‹ˆ๋‹ค.

์—์„œ

...
model-engine-file=model_b1_gpu0_fp32.engine
#int8-calib-file=calib.table
...
network-mode=0
...

To

...
model-engine-file=model_b1_gpu0_int8.engine
int8-calib-file=calib.table
...
network-mode=1
...
  • 9๋‹จ๊ณ„. ์ถ”๋ก  ์‹คํ–‰
deepstream-app -c deepstream_app_config.txt
YOLOv5 ๋”ฅ์ŠคํŠธ๋ฆผ INT8

์œ„์˜ ๊ฒฐ๊ณผ๋Š” INT8 ๋ฐ YOLOv5s 640x640์œผ๋กœ Jetson Xavier NX์—์„œ ์‹คํ–‰ํ•œ ๊ฒฐ๊ณผ์ž…๋‹ˆ๋‹ค. FPS๊ฐ€ 60 ์ •๋„์ž„์„ ์•Œ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

๋ฒค์น˜๋งˆํฌ ๊ฒฐ๊ณผ

๋‹ค์Œ ํ‘œ์—๋Š” Jetson Xavier NX์—์„œ ๋‹ค์–‘ํ•œ ๋ชจ๋ธ์˜ ์„ฑ๋Šฅ์ด ์š”์•ฝ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค.

๋ชจ๋ธ ์ด๋ฆ„ ์ •๋ฐ€๋„ ์ถ”๋ก  ํฌ๊ธฐ ์ถ”๋ก  ์‹œ๊ฐ„(ms) FPS
YOLOv5s FP32 320x320 16.66 60
FP32 640x640 33.33 30
INT8 640x640 16.66 60
YOLOv5n FP32 640x640 16.66 60

์ถ”๊ฐ€

์ด ํŠœํ† ๋ฆฌ์–ผ์€ @lakshanthad์™€ Elaine์˜ ์นœ๊ตฌ๋“ค์ด ์ž‘์„ฑํ–ˆ์Šต๋‹ˆ๋‹ค.



Created 2023-11-12, Updated 2024-06-10
Authors: glenn-jocher (7)

๋Œ“๊ธ€