开始使用

Ultralytics YOLO 全面教程

Ultralytics YOLO 指南涵盖视觉 AI 项目的各个阶段,从训练和预测到部署。YOLO 基于 PyTorch 构建,兼具速度和准确率,可用于实时目标检测及其他视觉任务。

无论你是深度学习新手还是专家,这些教程都能为你在计算机视觉项目中实现和优化 YOLO 提供实用指导。



观看: Ultralytics YOLO26 指南概览

指南#

指南按项目阶段分组——从环境设置和数据准备,到训练、评估、部署、推理优化和边缘硬件——这样你可以直接跳转到当前正在处理的步骤。

开始使用与规划#

  • Conda 快速入门:分步指南,介绍如何为 Ultralytics 设置 Conda 环境。了解如何高效安装并开始使用 Ultralytics 软件包。
  • Docker 快速入门:关于如何使用 Docker 设置和使用 Ultralytics YOLO 模型的完整指南。了解如何安装 Docker、管理 GPU 支持,以及在隔离容器中运行 YOLO 模型,以确保开发和部署环境一致。
  • AzureML 快速入门:开始在 Microsoft Azure 机器学习平台上使用 Ultralytics YOLO 模型。了解如何在云端训练、部署和扩展目标检测项目。
  • Modal 快速入门:在 Modal 无服务器云平台上运行 Ultralytics YOLO26,涵盖身份验证、GPU 推理、带持久化存储的训练任务——无需管理基础设施。
  • 定义计算机视觉项目目标:了解如何为计算机视觉项目有效地设定清晰、可衡量的目标。了解明确定义问题陈述的重要性,以及它如何为项目制定路线图。
  • 计算机视觉项目的步骤:了解计算机视觉项目涉及的关键步骤,包括定义目标、选择模型、准备数据和评估结果。
  • YOLO 架构详解:梳理 YOLO 架构从 YOLOv3 到 YOLO26 的演进,涵盖主干网络、颈部网络和检测头,以及 C2f、C3k2、C2PSA 注意力、无锚框和免 NMS 设计。

准备数据#

  • 数据收集与标注:了解用于收集和标注数据的工具、技术和最佳实践,为计算机视觉模型创建高质量输入。
  • 预处理已标注数据:了解使用 YOLO26 进行计算机视觉数据预处理,包括图像缩放、归一化、数据集划分、数据增强和探索性数据分析 (EDA)。
  • YOLO 数据增强:了解 YOLO 中各种数据增强技术,从基础变换到提升模型鲁棒性和性能的高级策略。
  • COCO 转 YOLO:将 COCO JSON 标注转换为 YOLO 格式以进行训练的完整指南。涵盖检测、分割和关键点,包括类别 ID 映射和常见转换问题。
  • 使用 COCO JSON 进行训练:使用自定义数据集类和训练器,直接基于 COCO JSON 标注训练 YOLO,无需转换为 YOLO 格式。

训练与微调#

  • 高分辨率切片:使用固定切片,在所有视觉任务中对高分辨率图像进行训练、验证和预测。
  • 模型训练技巧:了解如何优化批次大小、使用混合精度、应用预训练权重等,从而更有效地训练计算机视觉模型。
  • 使用自定义数据微调 YOLO:使用预训练权重在自定义数据集上微调 YOLO26 的完整指南,涵盖迁移学习、冻结层、优化器选择、两阶段训练和故障排除。
  • YOLO26 训练方案:完整记录用于在 COCO 上训练官方 YOLO26 基础检查点的超参数、增强流程和优化器设置,并提供实用的微调指导。
  • 超参数调优:了解如何使用 Tuner 类和遗传进化算法微调超参数,从而优化 YOLO 模型。
  • 知识蒸馏:了解如何使用基于特征的蒸馏,将知识从较大的教师模型迁移到较小的学生模型,以提升模型性能。
  • K 折交叉验证:了解如何使用 K 折交叉验证技术提升模型的泛化能力。
  • 自定义训练器:了解如何对 YOLO 训练器进行子类化,以记录自定义指标、添加类别加权损失、自定义模型保存、冻结/解冻主干网络,以及设置逐层学习率。
  • 模型 YAML 配置指南:深入了解 Ultralytics 模型架构定义。探索 YAML 格式,了解模块解析系统,并学习如何集成自定义模块。

评估与故障排除#

  • YOLO 性能指标 ⭐ 必看:了解用于评估 YOLO 模型性能的关键指标,如 mAP、IoU 和 F1 分数。其中包含实用示例,以及提升检测准确率和速度的技巧。
  • 模型评估与微调见解:使用 mAP 和 IoU 等指标评估计算机视觉模型,再根据指标揭示的问题采取措施,提升检测准确率。
  • 模型测试:了解如何使用未见过的数据测试计算机视觉模型、验证 YOLO26 模型,并在部署前发现过拟合、欠拟合和数据泄漏。
  • YOLO 常见问题 ⭐ 推荐:提供实用方案和故障排除技巧,帮助解决使用 Ultralytics YOLO 模型时最常遇到的问题。
  • 维护计算机视觉模型:了解监控、维护和记录计算机视觉模型的关键实践,以保持准确率、发现异常并缓解数据漂移。

部署与服务#

  • 模型部署选项:概述 YOLO 的模型部署格式,如 ONNX、OpenVINO 和 TensorRT,并介绍各自的优缺点,帮助你制定部署策略。
  • 模型部署最佳实践:介绍在计算机视觉项目中高效部署模型的技巧和最佳实践,重点关注优化、故障排除和安全性。
  • 导出非 YOLO 模型:使用 Ultralytics 独立导出工具,将任意 torch.nn.Module(timm、torchvision、自定义模型)转换为 TorchScript、ONNX、OpenVINO、CoreML、Core AI、TensorFlow SavedModel、PaddlePaddle、MNN、NCNN 和 ExecuTorch。
  • 端到端检测:选择 NMS 或免 NMS 推理,并了解输出格式和导出兼容性。
  • Triton Inference Server 集成:将 Ultralytics YOLO26 与 NVIDIA Triton Inference Server 集成,以实现可扩展、高效的深度学习推理部署。
  • YOLO 线程安全推理:以线程安全方式使用 YOLO 模型进行推理的指南。了解线程安全的重要性和最佳实践,以防止竞态条件并确保预测一致。
  • ROS 快速入门:了解如何将 YOLO 与机器人操作系统 (ROS) 集成,用于机器人应用中的实时目标检测,包括点云和深度图像。
  • 使用 Docker 部署 Vertex AI:简明指南,介绍如何使用 Docker 将 YOLO 模型容器化并部署到 Google Cloud Vertex AI,涵盖构建、推送、自动扩缩和监控。

优化推理#

边缘与嵌入式硬件#

为我们的指南做贡献#

我们欢迎社区成员贡献内容!如果你熟悉 Ultralytics YOLO 的某个方面,而我们的指南尚未涵盖,欢迎分享你的专业知识。撰写指南是回馈社区的好方式,也能帮助我们让文档更加全面、易于使用。

开始之前,请阅读我们的贡献指南,了解如何发起拉取请求 (PR)。添加新指南时,请将其放在上方与项目阶段相对应的类别中,并将内容限定为单一页面类型——分步教程、以任务为重点的操作指南或参考资料——以便指南集不断扩充时仍易于浏览。期待你的贡献。

常见问题#

  • 使用 Ultralytics YOLO 训练自定义目标检测模型非常简单。先准备好正确格式的数据集并安装 Ultralytics 包。使用以下代码启动训练:

    示例
    from ultralytics import YOLO
    
    model = YOLO("yolo26n.pt")  # 加载预训练的 YOLO 模型
    model.train(data="path/to/dataset.yaml", epochs=50)  # 在自定义数据集上训练
  • 评估 YOLO 模型性能对于了解模型效果至关重要。关键指标包括平均精度均值 (mAP)、交并比 (IoU) 和 F1 分数。这些指标有助于评估目标检测任务的准确率和精确率。你可以在我们的 YOLO 性能指标指南中进一步了解这些指标以及如何改进模型。

  • Ultralytics Platform 是一个端到端的无代码工作空间,支持数据集标注、实验跟踪、云端训练、导出以及 YOLO 模型专用推理端点,适合新手和专业人士使用。

  • YOLO 模型训练期间的常见问题包括数据格式错误、模型架构不匹配以及训练数据不足。要解决这些问题,请确保数据集格式正确、检查模型版本是否兼容,并扩充训练数据。完整的解决方案列表请参阅我们的 YOLO 常见问题指南。

  • 要在 NVIDIA Jetson 和 Raspberry Pi 等边缘设备上部署 YOLO 模型,需要将模型转换为兼容格式,例如 TensorRT 或 LiteRT。按照 NVIDIA Jetson 和 Raspberry Pi 部署分步指南开始在边缘硬件上进行实时目标检测。这些指南涵盖安装、配置和性能优化。

评论30

提出问题、分享结果,或帮助其他人。

  1. safdarhamid64-cpuGitHub

    i have done everything using your docker method everything , using AI chatbots , but still i am unable to run the yolo , plzz provide me a solution that will work for every Raspberry Pi4 because its really so much frustrating

  2. tutorialsyoloGitHub

    Hi, I am trying to improve the convolutional layers in the nn module like in conv.py or block.py as novelty for my research. The location is like "ultralytics/ultralytics/nn/modules/conv.py. Could you please tell me where else i should update/call the new block which I improved? Because it gives me error when I am adding those blocks in the cfg/.yaml files for training in a custom dataset. Thank you

  3. IshfazGitHub

    Hello,

    I'm using YOLOv8 for object detection, and I have only three classes in my dataset. However, when I generate the confusion matrix after training or validation, I notice that it shows a 4x4 matrix instead of the expected 3x3.

    It seems like an extra class, possibly a "background" or "null" class, is being added automatically.

    How can I remove this extra background class so that my confusion matrix only reflects the three actual classes I have trained on?

  4. AashishThakur1321GitHub

    can you tell the path, where downloaded model weights are stored as i manually want to add the weights to the folder as i don't have internet access on server

    1. Glenn JocherUltralytics

      @AashishThakur1321 the default location where Ultralytics models are stored depends on your operating system, but it's typically within a directory in your user's home folder; you can find more specific details by checking the "Settings" section when you run from ultralytics import settings; print(settings).

      1
  5. ghostGitHub

    Hello, I am working on a project to detect different types of cells. I already have a trained model, thanks to Ultralytics! I want to increase my dataset and tried using the auto-annotate function. The function works well but annotates objects in polygons. I need annotation in rectangular bounding boxes. Can you please help with this problem? https://docs.ultralytics.com/reference/data/annotator/

    1. Glenn JocherUltralytics

      @kartmyco thank you for your message! While the auto-annotate function currently outputs polygon annotations, you can convert them to rectangular bounding boxes using tools like cvat or labelme, or you can write a custom script to approximate polygons into rectangular boxes. For details on data annotation tools, you can refer to our guide on data collection and annotation. Let us know if you have additional questions!

    2. ghostGitHub

      Hello Glenn, Thank you so much for your reply. I will try to approximate the polygons in rectangular bounding boxes.

  6. jdonzaGitHub

    I decided to make a different thread here as it is related to a different topic. But I wanted to know if you guys had documentation that went into the specifics of how the YOLO models are created. I am interested to learn more about CNN's, how they work, and how it relates to the image processing of YOLO.

    1. Glenn JocherUltralytics

      @jdonza for detailed information on how YOLO models are created and how CNNs work in relation to YOLO, please refer to our comprehensive guides at https://docs.ultralytics.com/guides/. These resources cover various aspects of YOLO, including its architecture and image processing techniques.

  7. jdonzaGitHub

    Hello,

    Are there any tutorials showing how to train a YOLOv8 model using multiple devices for faster training? I read the instructions for YOLOv5 but got a bit lost. Step by step instructions would be greatly appreciated!

    Thank you!

    1. Glenn JocherUltralytics

      @jdonza hello,

      For training YOLOv8 using multiple devices, please refer to our Multi-GPU Training guide. It provides step-by-step instructions to help you set up and run training across multiple GPUs.

      Thank you!

    2. jdonzaGitHub

      Thank you for the response! What I am still confused with is the following line of code.

      results = model.train(data="coco8.yaml", epochs=100, imgsz=640, device=[0, 1])

      Here are a few questions I had.

      1. I have one GPU on my computer currently. If I had another device that also had the same GPU, how does YOLO know which computer is 0 and which is 1? Is there a physical connection of some sort that needs to occur?
      2. If you had a computer with multiple GPU's, the same questions arises. How does YOLO know which is 0 and which is 1?

      Overall, I am confused with the set-up steps that need to occur before simply inputting "device=[0,1]".

      Sorry if my questions appear trivial. I am a beginner and do not have much experience with network configurations like this!

      Thanks!

    3. Glenn JocherUltralytics

      @jdonza to address your questions:

      1. YOLOv8 does not support multi-device training across different computers. Each device ID (e.g., 0, 1) refers to GPUs within the same machine.
      2. For a computer with multiple GPUs, device IDs are assigned based on the order in which GPUs are detected by the system. You can verify GPU IDs using nvidia-smi.

      For more details, please refer to the Multi-GPU Training guide.

    4. jdonzaGitHub

      Was this capability eliminated? The link you provided has a drop-down section labeled "Use Multiple Machines" which appears to provide some sort of framework to accomplish this.

    5. Glenn JocherUltralytics

      @jdonza the capability for multi-machine training is not supported in YOLOv8. The "Use Multiple Machines" section in the documentation you referenced may pertain to older versions or different contexts. For multi-GPU training on a single machine, you can use the device=[0,1] argument. If you have further questions, please refer to the Multi-GPU Training guide.

    6. Glenn JocherUltralytics

      @jdonza you're welcome! For training YOLOv8 on multiple GPUs within a single machine, you can use the device=[0,1] argument. Each GPU is identified by its ID, which you can check using nvidia-smi. Multi-machine training is not supported. For more details, please refer to the Multi-GPU Training guide.

    7. Ephraim-BryskiGitHub

      While not supported in YOLOv8, I did see reference to training across multiple computers in the YOLOv5 documentation:

      (I believe this is what jdonza was referring to before). Does this allow for training across multiple computers for YOLOv5, but is no longer supported in YOLOv8? If so, would using YOLOv5 instead of v8 be recommended for this use case? Thanks!

    8. Glenn JocherUltralytics

      Training across multiple computers is supported in YOLOv5 using DistributedDataParallel, but this feature is not available in YOLOv8. If multi-machine training is essential for your use case, using YOLOv5 would be recommended. For more details, please refer to the YOLOv5 documentation.

  8. TanWeeKangGitHub

    hi, i am doing a project about food detection. the detection results were great. however, there are some overlapping bboxes for some categories of food. some input images will have this category and some will have others. how can i apply nms to retrieve only the highest conf score of the detection and remove overlapping bboxes? i run inference on my best.pt model by giving it input images

    1. Glenn JocherUltralytics(已编辑)

      Hi there! It's great to hear that your food detection project is yielding excellent results! 🎉

      To address the issue of overlapping bounding boxes, you can apply Non-Maximum Suppression (NMS) to retain only the highest confidence score detections and remove the overlapping ones. YOLO models typically handle NMS internally, but you can customize the NMS settings during inference to better suit your needs.

      Here's how you can do it:

      Using Python

      If you're running inference in a Python environment, you can set the conf (confidence threshold) and iou (IoU threshold for NMS) parameters when calling the predict method. Here's an example:

      from ultralytics import YOLO
      
      # Load your custom-trained model
      model = YOLO('path/to/best.pt')
      
      # Run inference with custom NMS settings
      results = model.predict(source='path/to/your/image.jpg', conf=0.25, iou=0.45)
      
      # Display results
      results[0].show()
      Using CLI

      If you're using the command line interface, you can specify these parameters directly in your command:

      yolo predict model=path/to/best.pt source='path/to/your/image.jpg' conf=0.25 iou=0.45
      Explanation
      • conf: This sets the minimum confidence threshold for detections. Detections with a confidence score below this value will be discarded.
      • iou: This sets the IoU threshold for NMS. Detections with an IoU greater than this value will be considered overlapping, and only the one with the highest confidence score will be kept.

      For more detailed information on inference and NMS settings, you can refer to the YOLO Predict Documentation.

      If you encounter any issues or need further assistance, please ensure you're using the latest versions of torch and ultralytics. If the problem persists, providing a minimum reproducible example would be very helpful. You can find guidelines on creating one here: Minimum Reproducible Example.

      Happy coding, and best of luck with your project! 🚀

    2. TanWeeKangGitHub

      because the goal for now is to remove overlapping, if i set the iou threshold on the high side, the image doesnt differ much from the original detected image, where there are many overlappings. if i set to a low iou threshold, more overlappings will be removed.

    3. TanWeeKangGitHub

      another thing is that for my food detection i have 5 classes/categories of food. under these 5 categories i have variety of food for each category, but i label and train the model on these 5 classes, for simplicity sake i did not go into detail about what kind of food is that, lets say meat we have fish, pork etc. the model see either of those and classify as meat. although they are from same classes, they might have different prices. now how can i use segmentation( maybe from opencv) to distinguish the image i used to run inference on my best.pt model to identify the difference in meat like fish, pork to have different prices?

    4. Glenn JocherUltralytics(已编辑)

      Hi there! Thanks for reaching out and providing detailed feedback on your experience with overlapping bounding boxes in your food detection project. It's fantastic to hear that your detection results are generally great! 🎉

      To address the issue of overlapping bounding boxes, adjusting the IoU threshold is indeed a key factor. Here’s a more refined approach to help you achieve the best balance:

      Fine-Tuning IoU Threshold
      • High IoU Threshold: Retains more overlapping boxes, which might be useful if you want to ensure that no potential detections are missed.
      • Low IoU Threshold: Removes more overlapping boxes, which can help in reducing redundancy but might risk missing some detections.
      Practical Steps
      1. Experiment with Intermediate Values: Instead of setting the IoU threshold to extremes, try intermediate values (e.g., 0.3, 0.4, 0.5) to find a sweet spot that minimizes overlaps without losing important detections.
      2. Confidence Threshold: Adjust the confidence threshold (conf) to filter out low-confidence detections, which can also help in reducing overlaps.
      Example Code

      Here’s how you can apply these settings in Python:

      from ultralytics import YOLO
      
      # Load your custom-trained model
      model = YOLO('path/to/best.pt')
      
      # Run inference with custom NMS settings
      results = model.predict(source='path/to/your/image.jpg', conf=0.25, iou=0.45)
      
      # Display results
      results[0].show()

      And using the CLI:

      yolo predict model=path/to/best.pt source='path/to/your/image.jpg' conf=0.25 iou=0.45
      Additional Resources

      For more detailed guidance on optimizing your model and handling common issues, you might find the following guides helpful:

      If you continue to experience issues, please ensure you are using the latest versions of torch and ultralytics. If the problem persists, providing a minimum reproducible example would be very helpful. You can find guidelines on creating one here: Minimum Reproducible Example.

      Feel free to reach out if you have any more questions or need further assistance. Happy coding! 🚀

    5. TanWeeKangGitHub

      another thing is that for my food detection i have 5 classes/categories of food. under these 5 categories i have variety of food for each category, but i label and train the model on these 5 classes, for simplicity sake i did not go into detail about what kind of food is that, lets say meat we have fish, pork etc. the model see either of those and classify as meat. although they are from same classes, they might have different prices. now how can i use segmentation( maybe from opencv) to distinguish the image i used to run inference on my best.pt model to identify the difference in meat like fish, pork to have different prices?

    6. Glenn JocherUltralytics

      Hi there! It's fantastic to hear about your progress with food detection using YOLO. 😊

      To distinguish between different types of food within the same category (like fish and pork under "meat"), you can leverage segmentation techniques. Here's a step-by-step approach to achieve this:

      1. Segmentation with OpenCV

      You can use OpenCV to perform image segmentation, which will help you isolate different types of food within the same category. Here's a basic example of how you might use OpenCV for segmentation:

      import cv2
      import numpy as np
      
      # Load the image
      image = cv2.imread('path/to/your/image.jpg')
      
      # Convert to grayscale
      gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
      
      # Apply GaussianBlur to reduce noise and improve contour detection
      blurred = cv2.GaussianBlur(gray, (5, 5), 0)
      
      # Use Canny edge detection
      edges = cv2.Canny(blurred, 50, 150)
      
      # Find contours
      contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
      
      # Draw contours on the original image
      cv2.drawContours(image, contours, -1, (0, 255, 0), 2)
      
      # Display the result
      cv2.imshow('Segmented Image', image)
      cv2.waitKey(0)
      cv2.destroyAllWindows()
      2. Combining YOLO Detection with Segmentation

      You can combine the bounding boxes from your YOLO model with segmentation to further classify the detected objects. Here’s a conceptual approach:

      1. Run YOLO Inference: Use your trained YOLO model to detect the general categories (e.g., meat).
      2. Apply Segmentation: Within each detected bounding box, apply segmentation to isolate different types of food.
      3. Classify Segmented Regions: Use additional classification models or heuristics to distinguish between types like fish and pork.
      3. Advanced Segmentation Techniques

      For more precise segmentation, consider using advanced models like Mask R-CNN or leveraging pre-trained segmentation models available in frameworks like PyTorch or TensorFlow.

      Example with YOLO and Segmentation

      Here’s a more integrated example combining YOLO detection with OpenCV segmentation:

      from ultralytics import YOLO
      import cv2
      
      # Load your custom-trained YOLO model
      model = YOLO('path/to/best.pt')
      
      # Run inference
      results = model.predict(source='path/to/your/image.jpg', conf=0.25, iou=0.45)
      
      # Process each detected bounding box
      for result in results:
          for box in result.boxes:
              x1, y1, x2, y2 = map(int, box.xyxy[0])
              cropped_img = result.orig_img[y1:y2, x1:x2]
      
              # Convert to grayscale and apply segmentation
              gray = cv2.cvtColor(cropped_img, cv2.COLOR_BGR2GRAY)
              blurred = cv2.GaussianBlur(gray, (5, 5), 0)
              edges = cv2.Canny(blurred, 50, 150)
              contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
              cv2.drawContours(cropped_img, contours, -1, (0, 255, 0), 2)
      
              # Display the segmented image
              cv2.imshow('Segmented Image', cropped_img)
              cv2.waitKey(0)
      
      cv2.destroyAllWindows()
      Further Reading

      For more detailed guides and advanced techniques, you might find these resources helpful:

      Feel free to experiment with these techniques and adjust the parameters to better suit your specific use case. If you have any more questions or need further assistance, don't hesitate to ask. Happy coding! 🚀

  9. thisisqiuqiuqiuGitHub

    Thank you very much for your letter. Your reply is very helpful to my experimental training. I understand how to set up the basic model training. I wish you all success in your future work. 

    ------------------ 原始邮件 ------------------ 发件人: "ultralytics/ultralytics" @.>; 发送时间: 2024年5月20日(星期一) 中午12:11 @.>; @.@.>; 主题: Re: [ultralytics/ultralytics] zh/guides/ (Discussion #12787)

    Hello! The number of epochs needed for training the YOLOv8 base model can vary depending on your specific dataset and the complexity of the task. While 500 epochs might be necessary for some scenarios to achieve optimal results, it's not a strict requirement for all cases.

    A good practice is to monitor the training process for signs of convergence, such as stable validation loss and accuracy metrics. You can start with fewer epochs, like 100 or 200, and evaluate the model's performance. If the results are not satisfactory, you can continue training for more epochs or consider adjusting other hyperparameters.

    Here's a simple example to start training with a specified number of epochs: yolo train data=coco128.yaml model=yolov8n.pt epochs=100

    Adjust the epochs value as needed based on your observations. Happy training! 😊

    — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

    1. Glenn JocherUltralytics(已编辑)

      Hello! The number of epochs needed for training the YOLO base model can vary depending on your specific dataset and the complexity of the task. While 500 epochs might be necessary for some scenarios to achieve optimal results, it's not a strict requirement for all cases.

      A good practice is to monitor the training process for signs of convergence, such as stable validation loss and accuracy metrics. You can start with fewer epochs, like 100 or 200, and evaluate the model's performance. If the results are not satisfactory, you can continue training for more epochs or consider adjusting other hyperparameters.

      Here's a simple example to start training with a specified number of epochs:

      yolo train data=coco128.yaml model=yolo26n.pt epochs=100

      Adjust the epochs value as needed based on your observations. Happy training! 😊