์ฝ˜ํ…์ธ ๋กœ ๊ฑด๋„ˆ๋›ฐ๊ธฐ

Optimizing YOLO11 Inferences with Neural Magic's DeepSparse Engine

When deploying object detection models like Ultralytics YOLO11 on various hardware, you can bump into unique issues like optimization. This is where YOLO11's integration with Neural Magic's DeepSparse Engine steps in. It transforms the way YOLO11 models are executed and enables GPU-level performance directly on CPUs.

This guide shows you how to deploy YOLO11 using Neural Magic's DeepSparse, how to run inferences, and also how to benchmark performance to ensure it is optimized.

Neural Magic์˜ ๋”ฅ์ŠคํŽ˜์ด์Šค

Neural Magic์˜ ๋”ฅ์ŠคํŽ˜์ด์Šค ๊ฐœ์š”

Neural Magic's DeepSparse is an inference run-time designed to optimize the execution of neural networks on CPUs. It applies advanced techniques like sparsity, pruning, and quantization to dramatically reduce computational demands while maintaining accuracy. DeepSparse offers an agile solution for efficient and scalable neural network execution across various devices.

Benefits of Integrating Neural Magic's DeepSparse with YOLO11

DeepSparse๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ YOLOV8 ๋ฅผ ๋ฐฐํฌํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์‚ดํŽด๋ณด๊ธฐ ์ „์—, DeepSparse ์‚ฌ์šฉ์˜ ์ด์ ์„ ์ดํ•ดํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค. ๋ช‡ ๊ฐ€์ง€ ์ฃผ์š” ์ด์ ์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค:

  • Enhanced Inference Speed: Achieves up to 525 FPS (on YOLO11n), significantly speeding up YOLO11's inference capabilities compared to traditional methods.

ํ–ฅ์ƒ๋œ ์ถ”๋ก  ์†๋„

  • Optimized Model Efficiency: Uses pruning and quantization to enhance YOLO11's efficiency, reducing model size and computational requirements while maintaining accuracy.

์ตœ์ ํ™”๋œ ๋ชจ๋ธ ํšจ์œจ์„ฑ

  • ํ‘œ์ค€ CPU์—์„œ ๊ณ ์„ฑ๋Šฅ: CPU์—์„œ GPU-์™€ ๊ฐ™์€ ์„ฑ๋Šฅ์„ ์ œ๊ณตํ•˜์—ฌ ๋‹ค์–‘ํ•œ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์— ๋ณด๋‹ค ์ ‘๊ทผํ•˜๊ธฐ ์‰ฝ๊ณ  ๋น„์šฉ ํšจ์œจ์ ์ธ ์˜ต์…˜์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.

  • Streamlined Integration and Deployment: Offers user-friendly tools for easy integration of YOLO11 into applications, including image and video annotation features.

  • Support for Various Model Types: Compatible with both standard and sparsity-optimized YOLO11 models, adding deployment flexibility.

  • ๋น„์šฉ ํšจ์œจ์ ์ด๊ณ  ํ™•์žฅ ๊ฐ€๋Šฅํ•œ ์†”๋ฃจ์…˜: ์šด์˜ ๋น„์šฉ์„ ์ ˆ๊ฐํ•˜๊ณ  ๊ณ ๊ธ‰ ๊ฐ์ฒด ํƒ์ง€ ๋ชจ๋ธ์˜ ํ™•์žฅ ๊ฐ€๋Šฅํ•œ ๋ฐฐํฌ๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.

Neural Magic ์˜ ๋”ฅ์ŠคํŽ˜์ด์Šค ๊ธฐ์ˆ ์€ ์–ด๋–ป๊ฒŒ ์ž‘๋™ํ•˜๋‚˜์š”?

Neural Magic์˜ ๋”ฅ ์ŠคํŒŒ์Šค ๊ธฐ์ˆ ์€ ์ธ๊ฐ„ ๋‘๋‡Œ์˜ ์‹ ๊ฒฝ๋ง ๊ณ„์‚ฐ ํšจ์œจ์„ฑ์—์„œ ์˜๊ฐ์„ ๋ฐ›์•˜์Šต๋‹ˆ๋‹ค. ์ด ๊ธฐ์ˆ ์€ ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๋‘ ๊ฐ€์ง€ ๋‡Œ์˜ ํ•ต์‹ฌ ์›๋ฆฌ๋ฅผ ์ฑ„ํƒํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค:

  • Sparsity: The process of sparsification involves pruning redundant information from deep learning networks, leading to smaller and faster models without compromising accuracy. This technique reduces the network's size and computational needs significantly.

  • ์ฐธ์กฐ ์œ„์น˜: DeepSparse๋Š” ๋„คํŠธ์›Œํฌ๋ฅผ Tensor ์—ด๋กœ ๋ถ„ํ• ํ•˜๋Š” ๊ณ ์œ ํ•œ ์‹คํ–‰ ๋ฐฉ๋ฒ•์„ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ์—ด์€ CPU ์˜ ์บ์‹œ ๋‚ด์— ์™„์ „ํžˆ ๋“ค์–ด๋งž๋„๋ก ๊นŠ์ด ๋‹จ์œ„๋กœ ์‹คํ–‰๋ฉ๋‹ˆ๋‹ค. ์ด ์ ‘๊ทผ ๋ฐฉ์‹์€ ๋‘๋‡Œ์˜ ํšจ์œจ์„ฑ์„ ๋ชจ๋ฐฉํ•˜์—ฌ ๋ฐ์ดํ„ฐ ์ด๋™์„ ์ตœ์†Œํ™”ํ•˜๊ณ  CPU ์˜ ์บ์‹œ ์‚ฌ์šฉ์„ ์ตœ๋Œ€ํ™”ํ•ฉ๋‹ˆ๋‹ค.

Neural Magic ์˜ ๋”ฅ์ŠคํŽ˜์–ด์Šค ๊ธฐ์ˆ  ์ž‘๋™ ๋ฐฉ์‹

Neural Magic ์˜ ๋”ฅ์ŠคํŽ˜์ด์Šค ๊ธฐ์ˆ  ์ž‘๋™ ๋ฐฉ์‹์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํ•ด๋‹น ๋ธ”๋กœ๊ทธ ๊ฒŒ์‹œ๋ฌผ์„ ์ฐธ์กฐํ•˜์„ธ์š”.

Creating A Sparse Version of YOLO11 Trained on a Custom Dataset

SparseZoo, an open-source model repository by Neural Magic, offers a collection of pre-sparsified YOLO11 model checkpoints. With SparseML, seamlessly integrated with Ultralytics, users can effortlessly fine-tune these sparse checkpoints on their specific datasets using a straightforward command-line interface.

Checkout Neural Magic's SparseML YOLO11 documentation for more details.

์‚ฌ์šฉ๋ฒ•: DeepSparse๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ YOLOV8 ๋ฐฐํฌ

Deploying YOLO11 with Neural Magic's DeepSparse involves a few straightforward steps. Before diving into the usage instructions, be sure to check out the range of YOLO11 models offered by Ultralytics. This will help you choose the most appropriate model for your project requirements. Here's how you can get started.

1๋‹จ๊ณ„: ์„ค์น˜

ํ•„์š”ํ•œ ํŒจํ‚ค์ง€๋ฅผ ์„ค์น˜ํ•˜๋ ค๋ฉด ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค:

์„ค์น˜

# Install the required packages
pip install deepsparse[yolov8]

Step 2: Exporting YOLO11 to ONNX Format

DeepSparse Engine requires YOLO11 models in ONNX format. Exporting your model to this format is essential for compatibility with DeepSparse. Use the following command to export YOLO11 models:

๋ชจ๋ธ ๋‚ด๋ณด๋‚ด๊ธฐ

# Export YOLO11 model to ONNX format
yolo task=detect mode=export model=yolo11n.pt format=onnx opset=13

์ด ๋ช…๋ น์€ yolo11n.onnx ๋ชจ๋ธ์„ ๋””์Šคํฌ์— ์ €์žฅํ•ฉ๋‹ˆ๋‹ค.

3๋‹จ๊ณ„: ์ถ”๋ก  ๋ฐฐํฌ ๋ฐ ์‹คํ–‰

With your YOLO11 model in ONNX format, you can deploy and run inferences using DeepSparse. This can be done easily with their intuitive Python API:

์ถ”๋ก  ๋ฐฐํฌ ๋ฐ ์‹คํ–‰

from deepsparse import Pipeline

# Specify the path to your YOLO11 ONNX model
model_path = "path/to/yolo11n.onnx"

# Set up the DeepSparse Pipeline
yolo_pipeline = Pipeline.create(task="yolov8", model_path=model_path)

# Run the model on your images
images = ["path/to/image.jpg"]
pipeline_outputs = yolo_pipeline(images=images)

4๋‹จ๊ณ„: ์„ฑ๋Šฅ ๋ฒค์น˜๋งˆํ‚น

It's important to check that your YOLO11 model is performing optimally on DeepSparse. You can benchmark your model's performance to analyze throughput and latency:

๋ฒค์น˜๋งˆํ‚น

# Benchmark performance
deepsparse.benchmark model_path="path/to/yolo11n.onnx" --scenario=sync --input_shapes="[1,3,640,640]"

5๋‹จ๊ณ„: ์ถ”๊ฐ€ ๊ธฐ๋Šฅ

DeepSparse provides additional features for practical integration of YOLO11 in applications, such as image annotation and dataset evaluation.

์ถ”๊ฐ€ ๊ธฐ๋Šฅ

# For image annotation
deepsparse.yolov8.annotate --source "path/to/image.jpg" --model_filepath "path/to/yolo11n.onnx"

# For evaluating model performance on a dataset
deepsparse.yolov8.eval --model_path "path/to/yolo11n.onnx"

์ฃผ์„ ๋‹ฌ๊ธฐ ๋ช…๋ น์„ ์‹คํ–‰ํ•˜๋ฉด ์ง€์ •๋œ ์ด๋ฏธ์ง€๊ฐ€ ์ฒ˜๋ฆฌ๋˜๊ณ , ๊ฐ์ฒด๋ฅผ ๊ฐ์ง€ํ•˜๊ณ , ์ฃผ์„์ด ๋‹ฌ๋ฆฐ ์ด๋ฏธ์ง€๊ฐ€ ๊ฒฝ๊ณ„ ์ƒ์ž ๋ฐ ๋ถ„๋ฅ˜์™€ ํ•จ๊ป˜ ์ €์žฅ๋ฉ๋‹ˆ๋‹ค. ์ฃผ์„์ด ๋‹ฌ๋ฆฐ ์ด๋ฏธ์ง€๋Š” ์ฃผ์„ ๊ฒฐ๊ณผ ํด๋”์— ์ €์žฅ๋ฉ๋‹ˆ๋‹ค. ์ด๋ ‡๊ฒŒ ํ•˜๋ฉด ๋ชจ๋ธ์˜ ๊ฐ์ง€ ๊ธฐ๋Šฅ์„ ์‹œ๊ฐ์ ์œผ๋กœ ํ‘œํ˜„ํ•˜๋Š” ๋ฐ ๋„์›€์ด ๋ฉ๋‹ˆ๋‹ค.

์ด๋ฏธ์ง€ ์ฃผ์„ ๊ธฐ๋Šฅ

After running the eval command, you will receive detailed output metrics such as precision, recall, and mAP (mean Average Precision). This provides a comprehensive view of your model's performance on the dataset. This functionality is particularly useful for fine-tuning and optimizing your YOLO11 models for specific use cases, ensuring high accuracy and efficiency.

์š”์•ฝ

This guide explored integrating Ultralytics' YOLO11 with Neural Magic's DeepSparse Engine. It highlighted how this integration enhances YOLO11's performance on CPU platforms, offering GPU-level efficiency and advanced neural network sparsity techniques.

For more detailed information and advanced usage, visit Neural Magic's DeepSparse documentation. Also, check out Neural Magic's documentation on the integration with YOLO11 here and watch a great session on it here.

Additionally, for a broader understanding of various YOLO11 integrations, visit the Ultralytics integration guide page, where you can discover a range of other exciting integration possibilities.

์ž์ฃผ ๋ฌป๋Š” ์งˆ๋ฌธ

What is Neural Magic's DeepSparse Engine and how does it optimize YOLO11 performance?

Neural Magic's DeepSparse Engine is an inference runtime designed to optimize the execution of neural networks on CPUs through advanced techniques such as sparsity, pruning, and quantization. By integrating DeepSparse with YOLO11, you can achieve GPU-like performance on standard CPUs, significantly enhancing inference speed, model efficiency, and overall performance while maintaining accuracy. For more details, check out the Neural Magic's DeepSparse section.

How can I install the needed packages to deploy YOLO11 using Neural Magic's DeepSparse?

Installing the required packages for deploying YOLO11 with Neural Magic's DeepSparse is straightforward. You can easily install them using the CLI. Here's the command you need to run:

pip install deepsparse[yolov8]

Once installed, follow the steps provided in the Installation section to set up your environment and start using DeepSparse with YOLO11.

How do I convert YOLO11 models to ONNX format for use with DeepSparse?

To convert YOLO11 models to the ONNX format, which is required for compatibility with DeepSparse, you can use the following CLI command:

yolo task=detect mode=export model=yolo11n.pt format=onnx opset=13

This command will export your YOLO11 model (yolo11n.pt)๋ฅผ ํ˜•์‹(yolo11n.onnx)๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋”ฅ์ŠคํŽ˜์ด์Šค ์—”์ง„์—์„œ ํ™œ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋ชจ๋ธ ๋‚ด๋ณด๋‚ด๊ธฐ์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ ๋ชจ๋ธ ๋‚ด๋ณด๋‚ด๊ธฐ ์„น์…˜.

How do I benchmark YOLO11 performance on the DeepSparse Engine?

Benchmarking YOLO11 performance on DeepSparse helps you analyze throughput and latency to ensure your model is optimized. You can use the following CLI command to run a benchmark:

deepsparse.benchmark model_path="path/to/yolo11n.onnx" --scenario=sync --input_shapes="[1,3,640,640]"

์ด ๋ช…๋ น์€ ์ค‘์š”ํ•œ ์„ฑ๋Šฅ ๋ฉ”ํŠธ๋ฆญ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ์„ฑ๋Šฅ ๋ฒค์น˜๋งˆํ‚น ์„น์…˜์„ ์ฐธ์กฐํ•˜์„ธ์š”.

Why should I use Neural Magic's DeepSparse with YOLO11 for object detection tasks?

Integrating Neural Magic's DeepSparse with YOLO11 offers several benefits:

  • Enhanced Inference Speed: Achieves up to 525 FPS, significantly speeding up YOLO11's capabilities.
  • ์ตœ์ ํ™”๋œ ๋ชจ๋ธ ํšจ์œจ์„ฑ: ํฌ์†Œ์„ฑ, ๊ฐ€์ง€์น˜๊ธฐ, ์ •๋Ÿ‰ํ™” ๊ธฐ์ˆ ์„ ์‚ฌ์šฉํ•˜์—ฌ ์ •ํ™•๋„๋ฅผ ์œ ์ง€ํ•˜๋ฉด์„œ ๋ชจ๋ธ ํฌ๊ธฐ์™€ ๊ณ„์‚ฐ ์š”๊ตฌ ์‚ฌํ•ญ์„ ์ค„์ž…๋‹ˆ๋‹ค.
  • ํ‘œ์ค€ CPU์—์„œ ๋†’์€ ์„ฑ๋Šฅ: ๋น„์šฉ ํšจ์œจ์ ์ธ CPU ํ•˜๋“œ์›จ์–ด์—์„œ GPU-์™€ ๊ฐ™์€ ์„ฑ๋Šฅ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.
  • ๊ฐ„์†Œํ™”๋œ ํ†ตํ•ฉ: ๊ฐ„ํŽธํ•œ ๋ฐฐํฌ ๋ฐ ํ†ตํ•ฉ์„ ์œ„ํ•œ ์‚ฌ์šฉ์ž ์นœํ™”์ ์ธ ๋„๊ตฌ.
  • Flexibility: Supports both standard and sparsity-optimized YOLO11 models.
  • ๋น„์šฉ ํšจ์œจ์ : ํšจ์œจ์ ์ธ ๋ฆฌ์†Œ์Šค ํ™œ์šฉ์„ ํ†ตํ•ด ์šด์˜ ๋น„์šฉ์„ ์ ˆ๊ฐํ•ฉ๋‹ˆ๋‹ค.

For a deeper dive into these advantages, visit the Benefits of Integrating Neural Magic's DeepSparse with YOLO11 section.


9๊ฐœ์›” ์ „ ์ƒ์„ฑ๋จ โœ๏ธ ์—…๋ฐ์ดํŠธ๋จ 7์ผ ์ „

๋Œ“๊ธ€