开始使用

深入了解性能指标

简介#

性能指标是评估准确率和目标检测模型效率的关键工具。它们可以揭示模型识别和定位图像中对象的效果。此外,它们还能帮助你了解模型处理假阳性和假阴性的情况。这些信息对于评估和提升模型性能至关重要。本指南将介绍与 YOLO26 相关的性能指标、这些指标的重要性,以及如何解读这些指标。



观看: Ultralytics YOLO26 性能指标 | mAP、F1 分数、精确率、IoU 和准确率

目标检测指标#

以下指标对于 YOLO26 十分重要,也广泛适用于不同的目标检测模型。

  • 交并比 (IoU): IoU 用于量化预测边界框与真实边界框之间的重叠程度。它是评估目标定位精度的基础指标。

  • 平均精度 (AP): AP 计算精确率-召回率曲线下的面积,用单个数值概括模型的精确率和召回率表现。

  • 平均精度均值 (mAP): mAP 扩展了 AP 的概念,通过计算多个目标类别的 AP 均值来得出结果。该指标适用于多类别目标检测场景,可全面评估模型性能。

  • 精确率和召回率:精确率量化所有正向预测中真正例所占的比例,用于评估模型避免假阳性的能力。召回率计算所有实际正例中真正例所占的比例,用于衡量模型检测某个类别所有实例的能力。

  • F1 分数:F1 分数是精确率和召回率的调和平均值,在同时考虑假阳性和假阴性的情况下,均衡评估模型性能。

如何计算 YOLO26 模型的指标#

YOLO26 的验证模式会计算上述评估指标。

使用验证模式很简单。训练好模型后,调用 model.val() 函数。该函数随后会处理验证数据集并返回各种性能指标。但这些指标意味着什么?你又该如何解读它们?

解读输出#

以下各节将逐项解析 model.val() 输出的内容。

按类别划分的指标#

输出内容之一是按类别细分的性能指标。这些细粒度信息有助于你了解模型在每个特定类别上的表现,尤其是在对象类别多样的数据集中。对于数据集中的每个类别,都会提供以下信息:

  • 类别:物体类别的名称,例如“person”“car”或“dog”。

  • 图像数:验证集中包含该物体类别的图像数量。

  • 实例数:该类别在验证集所有图像中出现的总次数。

  • Box(P, R, mAP50, mAP50-95):该指标可帮助了解模型的目标检测性能:

    • P(精确率):检测结果的准确性,表示有多少检测结果是正确的。

    • R(召回率):模型识别图像中所有物体实例的能力。

    • mAP50:在交并比(IoU)阈值为 0.50 时计算的平均精度均值。该指标衡量模型的准确率,仅考虑“容易”检测的目标。

    • mAP50-95:在 0.50 到 0.95 的不同 IoU 阈值下计算平均精度均值,再取其平均值。该指标全面反映模型在不同检测难度下的性能。

速度指标#

推理速度可能与精度同样重要,尤其是在实时目标检测场景中。本节将分解验证过程各个阶段所耗费的时间,从预处理到后处理。

COCO 指标评估#

对于在 COCO 数据集上进行验证的用户,还会使用 COCO 评估脚本计算其他指标。这些指标可以揭示模型在不同 IoU 阈值以及不同尺寸对象上的精确率和召回率。

可视化输出#

model.val() 函数除了生成数值指标外,还会生成可视化输出,帮助你更直观地了解模型性能。以下是你可以看到的可视化输出:

  • F1 分数曲线 (BoxF1_curve.png):此曲线展示了不同阈值下的 F1 分数。解读此曲线有助于了解模型在不同阈值下平衡假阳性和假阴性的能力。

  • 精确率-召回率曲线 (BoxPR_curve.png):这是任何分类问题的重要可视化图表,展示不同阈值下精确率和召回率之间的权衡。在处理类别不平衡问题时,这条曲线尤为重要。

  • 精确率曲线 (BoxP_curve.png):以图形方式展示不同阈值下的精确率值。这条曲线有助于了解精确率如何随阈值变化。

  • 召回率曲线 (BoxR_curve.png):相应地,这张图展示了召回率如何随不同阈值变化。

  • 混淆矩阵 (confusion_matrix.png):混淆矩阵详细展示了各类别的结果,包括真阳性、真阴性、假阳性和假阴性的数量。

  • 归一化混淆矩阵 (confusion_matrix_normalized.png):这张图展示了归一化后的混淆矩阵,以比例而非原始计数表示数据。这种格式便于比较不同类别的性能。

  • 验证批次标签 (val_batchX_labels.jpg):这些图像展示了验证数据集中不同批次的真实标签,清晰呈现数据集中对象的类别及其对应位置。

  • 验证批次预测 (val_batchX_pred.jpg):与标签图像相对应,这些图像展示了 YOLO26 对相应批次作出的预测。将预测与标签图像进行比较,你可以直观地评估模型检测和分类对象的效果。

对于检测、分割和姿态任务,曲线图的前缀表示指标类型:检测任务生成 Box* 曲线,分割任务生成 Box* 和 Mask* 曲线,姿态任务生成 Box* 和 Pose* 曲线。

结果存储#

你可以留存这些结果以供日后参考;结果会保存到一个目录中,通常名为 runs/detect/val。

选择合适的指标#

选择合适的指标进行评估,通常取决于具体应用场景。

  • mAP: 适用于全面评估模型性能。

  • IoU: 在需要精确定位对象时至关重要。

  • 精确率: 当优先考虑减少误检时,这项指标很重要。

  • 召回率: 当需要检测出对象的每个实例时,这项指标至关重要。

  • F1 分数: 适用于需要平衡精确率和召回率的场景。

对于实时应用,FPS(每秒帧数)和延迟等速度指标至关重要,可确保及时获得结果。

解读结果#

以下是一些常见低分可能反映的情况:

  • mAP 较低: 表明模型可能需要进行整体优化。

  • IoU 较低: 模型可能难以准确定位对象。尝试使用不同的边界框方法或许会有所帮助。

  • 精确率较低: 模型可能检测出了太多不存在的对象。调整置信度阈值或许可以减少误检。

  • 召回率较低: 模型可能漏检了真实对象。改进特征提取或增加数据或许会有所帮助。

  • F1 分数不平衡: 精确率和召回率之间存在差距。

  • 特定类别的 AP: 这项指标得分较低,可能说明模型难以处理某些类别。

案例分析#

真实案例有助于说明这些指标在实际中的作用。

案例 1#

  • 情况:mAP 和 F1 分数不理想;召回率良好,但精确率不佳。

  • 解读与行动: 误检可能太多。收紧置信度阈值可以减少误检,但也可能略微降低召回率。

案例 2#

  • 情况: mAP 和召回率表现尚可,但 IoU 较低。

  • 解读与行动: 模型能够很好地检测对象,但定位可能不够精确。改进边界框预测或许会有所帮助。

案例 3#

  • 情况: 即使整体 mAP 不错,某些类别的 AP 仍远低于其他类别。

  • 解读与操作:这些类别对模型来说可能更难处理。增加这些类别的数据,或使用 cls_pw 训练参数调整类别权重,可能会有所帮助。

交流与协作#

YOLO26 社区可以通过以下渠道帮助你学习、排查问题和建立联系。

参与更广泛的社区交流#

  • GitHub Issues: Ultralytics 在 GitHub 上的代码库设有 Issues 标签页,你可以在这里提问、报告 Bug 和建议新功能。社区成员和维护者都积极参与交流,这也是针对具体问题寻求帮助的好地方。

  • Ultralytics Discord 服务器: Ultralytics 设有 Discord 服务器,你可以在那里与其他用户和开发者交流。

官方文档和资源#

  • Ultralytics YOLO26 文档: 官方文档全面介绍了 YOLO26,并提供安装、使用和故障排查指南。

这些资源可以帮助你解决问题,并了解 YOLO26 社区的最新最佳实践。

结论#

本指南详细介绍了 YOLO26 的关键性能指标。这些指标是了解模型表现的关键,对希望微调模型的人至关重要。它们能提供必要的参考信息,帮助你改进模型,并确保模型在实际场景中有效运行。

YOLO26 和 Ultralytics 社区也是实用资源:与其他开发者和专家交流,可能会找到文档未涵盖的解决方案;分享你的发现也能帮助其他从事目标检测工作的人。

常见问题#

  • 平均精度均值(mAP)对于评估 YOLO26 模型至关重要,因为它用一个指标概括了模型在多个类别上的精确率和召回率。mAP50 衡量 IoU 阈值为 0.50 时的精确率,侧重于模型正确检测目标的能力。mAP50-95则计算一系列 IoU 阈值下的平均精确率,全面评估目标检测性能。较高的 mAP 分数表明模型能有效平衡精确率和召回率,这对于自动驾驶和监控系统等应用至关重要,因为这些场景既要求准确检测,也要求尽量减少误报。

  • 交并比 (IoU) 用于衡量预测边界框与真实边界框之间的重叠程度。IoU 的取值范围为 0 到 1,值越高表示定位越准确。IoU 为 1.0 表示完全重合。通常,mAP 等指标会使用 0.50 作为 IoU 阈值来定义真阳性。IoU 值较低表明模型难以精确定位对象;你可以通过改进边界框回归,或提高训练数据集中的标注精度来改善这一问题。

  • F1 分数对于评估 YOLO26 模型很重要,因为它是精确率和召回率的调和平均值,能够平衡假阳性和假阴性。在处理类别不平衡的数据集,或单独使用精确率或召回率都不够的应用场景中,F1 分数尤其有用。F1 分数较高表明模型能够有效检测对象,同时尽量减少漏检和误报,因此适用于安全系统和医学影像等关键应用。

  • Ultralytics YOLO26 在实时对象检测方面具有多项优势:

    • 速度与效率:针对高速推理进行了优化,适用于对低延迟有要求的应用。
    • 高准确率:在不同模型规模下实现较高的 mAP,同时兼顾精确率和召回率。
    • 灵活性:支持多种任务,包括对象检测、实例分割、语义分割和分类。
    • 易于使用:提供用户友好的界面、详尽的文档,并可与 Ultralytics Platform 等工具集成。

    因此,YOLO26 非常适用于各种应用场景,从自动驾驶汽车到智慧城市解决方案。

  • YOLO26 的精确率、召回率、mAP 和 IoU 等验证指标可以从检测的不同方面提供参考,帮助你诊断并提升模型性能:

    • 精确率:有助于识别并减少假阳性。
    • 召回率:确保检测出所有相关对象。
    • mAP:概览整体性能,为全面改进提供参考。
    • IoU:有助于微调对象定位精度。

    分析这些指标可以帮助你找到具体的薄弱环节,例如调整置信度阈值以提高精确率,或收集更多样化的数据以提升召回率。如需详细了解这些指标及其解读方法,请参阅对象检测指标,并考虑进行超参数调优来优化模型。

评论109

提出问题、分享结果,或帮助其他人。

  1. atharvahudeGitHub

    Question: Clarification on AP vs mAP in YOLOv8 metrics output

    Hello,

    I'm using YOLOv8 for object detection, and I have some questions about the metrics output, particularly regarding the use of map50 and map50-90 in the results. In the output, I'm seeing these metrics reported for individual classes as well as an "all" category that aggregates the performance.

    My understanding is that:

    AP (Average Precision) should be calculated for each individual class.

    mAP (mean Average Precision) is the average of the AP values across all classes.

    However, in the output, the terms map50 and map50-90 are listed for each individual class. Should these values be interpreted as AP for each class, rather than mAP, which I would expect to be the mean across all classes? Additionally, the "all" row appears to be giving the mAP value.

    Could you confirm:

    1. Should the per-class map50 and map50-90 values be treated as AP for those specific classes?

    2. Is the "all" row correctly representing the mean AP across all classes?

    Thanks in advance for your clarification!

    1
    1. Glenn JocherUltralytics

      @atharvahude the per-class map50 and map50-95 values should indeed be treated as AP for those specific classes. The "all" row represents the mean AP across all classes, confirming your understanding. For more details, you can refer to our documentation on performance metrics.

  2. e-roeGitHub

    Hi, Great guide on performance metrics — they're key tools to evaluate the accuracy and efficiency of object detection models. However, although the plots are shown, I think the train/box_loss, train/obj_loss, and train/cls_loss plots weren't explained — or did I miss something? Thanks!

    1. Glenn JocherUltralytics(已编辑)

      @e-roe great observation! You're right that the training loss plots could use more explanation. The training losses shown during YOLO11 training represent:

      • train/box_loss: Localization loss measuring how well predicted bounding boxes match ground truth coordinates
      • train/cls_loss: Classification loss measuring how accurately the model predicts object classes
      • train/dfl_loss: Distribution Focal Loss, which helps improve localization accuracy by treating bounding box regression as a classification problem (YOLO26 is DFL-free, so this term is a plain L1 box-distance loss there)

      These training losses should generally decrease over epochs, indicating the model is learning. The validation losses (val/box_loss, val/cls_loss, val/dfl_loss) help monitor overfitting - if training losses decrease but validation losses increase or plateau, it suggests overfitting.

      Thanks for pointing this out! We'll consider expanding the training loss explanations in future documentation updates.

  3. Hemanth-mandava-neuropiGitHub

    @glenn-jocher I'm working on a YOLOv8 segmentation task to detect pallets. I've annotated a dataset using Roboflow with 6000 images, split into train, test, and validation sets. After training, I obtained the following results:

    Class Images Instances Box(P R mAP50 mAP50-95) Mask(P R mAP50 mAP50-95) all 1227 2262 0.974 0.489 0.689 0.636 0.972 0.488 0.690 0.624 The precision is high, but the recall is quite low (~0.48), which suggests many true pallets are being missed.

    While inspecting the Roboflow data analytics, I noticed something interesting:

    Around 4000 images have only 1 annotated pallet

    The rest have 2–10 pallets annotated

    This leads me to the following questions:

    Does this cause class imbalance in the dataset, particularly affecting model generalization and recall?

    Should I balance the dataset between single-object and multi-object images? Is it worth filtering or augmenting to even it out?

    Since I’m combining pre-annotated data from multiple sources, is there a way to analyze the distribution of annotations/images before training, especially when not using Roboflow’s UI?

    What techniques are most effective to boost recall in segmentation tasks? Should I look into:

    Lowering confidence thresholds?

    Data augmentation (mosaic, copy-paste)?

    Label smoothing or loss function tweaks?

    Different anchor/mask threshold settings?

    Any help or insights would be greatly appreciate

  4. Fatima-khanjarGitHub(已编辑)

    Hello!

    I hope this message finds you well.

    I have a question regarding the model's performance. Typically, to evaluate the model's performance, we look at precision, recall, and F1 scores. However, these metrics alone may not be sufficient to confirm that the model is performing well. We should also consider the values of DFL, classification (cls) loss, and box loss. My question is: What are the values of these losses that are considered good? Should they be something like 0.00xxxx? Additionally, what is the significance of having high precision, recall, and F1 scores (above 0.8), while at the same time, the loss values are around 0.3 and 0.6, and when we look at the plots of losses (train vs. val), we see that both losses are decreasing over the epochs?

    I have another question: The plots and values for training and validation losses are saved, but how can I extract the overall values of train/val losses during the training process?

    Thank you for your help!

    Regards,

    1. Glenn JocherUltralytics

      @Fatima-khanjar excellent questions about model evaluation!

      For DFL, classification, and box losses, there are no absolute "good" values as they depend on your dataset complexity, model size, and task. What's more important is the trend - consistently decreasing losses indicate proper training. Loss values of 0.3-0.6 with high precision/recall (>0.8) can be completely normal, as metrics often continue improving even after losses plateau.

      To extract training/validation loss values programmatically:

      import pandas as pd
      
      # Load the results CSV
      results_df = pd.read_csv('path/to/results.csv')
      
      # Extract train/val losses
      train_box_loss = results_df['train/box_loss']
      val_box_loss = results_df['val/box_loss']
      train_cls_loss = results_df['train/cls_loss']
      val_cls_loss = results_df['val/cls_loss']
      train_dfl_loss = results_df['train/dfl_loss']
      val_dfl_loss = results_df['val/dfl_loss']

      Focus on comparing validation metrics against your application requirements rather than trying to reach arbitrary loss values. A model with slightly higher losses but better real-world performance is ultimately more valuable.

      1
    2. maomao123-gitGitHub

      Hi!how to print f1 score in model.val() ? In addition, I want to print metrics with four decimal places so that I can get percentage form. For example, the printed mAP50 was 0.8657, and I can transfer the number to 86.57%. Where and how can I revise the code? Thanks!

    3. Glenn JocherUltralytics(已编辑)

      @maomao123-git to print the F1 score and format metrics with four decimal places when running validation, you can access these values from the results object:

      from ultralytics import YOLO
      
      # Load your model
      model = YOLO('yolo26n.pt')  # or your custom model path
      
      # Run validation
      results = model.val()
      
      # results.box.f1 holds one F1 score per class
      f1 = results.box.f1.mean()
      map50 = results.box.map50
      
      # Format and print
      print(f"F1 Score: {f1:.4f} ({f1*100:.2f}%)")
      print(f"mAP50: {map50:.4f} ({map50*100:.2f}%)")
      
      # For class-specific metrics
      for i, f1_val in enumerate(results.box.f1):
          class_name = model.names[results.box.ap_class_index[i]]
          print(f"Class '{class_name}' F1 Score: {f1_val:.4f} ({f1_val*100:.2f}%)")

      This gives you access to both overall and class-specific F1 scores with your desired formatting. The metrics class includes all the performance metrics like precision, recall, mAP, and F1 score, which you can format as needed.

      2
    4. Glenn JocherUltralytics(已编辑)

      @maomao123-git to get the F1 score for "all" classes (mean F1 across all classes), you can calculate it from the overall precision and recall values:

      # Run validation
      results = model.val()
      
      # Get overall precision and recall
      precision_all = results.box.mp  # mean precision
      recall_all = results.box.mr     # mean recall
      
      # Calculate F1 score for "all" classes
      f1_all = 2 * (precision_all * recall_all) / (precision_all + recall_all) if (precision_all + recall_all) > 0 else 0
      
      # Format and print
      print(f"All Classes F1 Score: {f1_all:.4f} ({f1_all*100:.2f}%)")

      Alternatively, you can calculate the mean of class-specific F1 scores:

      # Calculate mean of all class F1 scores
      mean_f1 = results.box.f1.mean()
      print(f"Mean F1 Score: {mean_f1:.4f} ({mean_f1*100:.2f}%)")

      Both methods should give you a good representation of the overall F1 score that would appear in the "all" row.

      2
    5. Glenn JocherUltralytics

      @Shubham77saini the precision calculation in YOLOv8 considers confidence thresholds and precision-recall curve interpolation, which can lead to different values than direct TP/(TP+FP) calculations. The reported 0.734 precision for the "car" class is likely affected by how the model handles confidence scores across the precision-recall curve.

      To understand this better, you might want to:

      1. Check the confusion matrix at different confidence thresholds
      2. Look at P-R curves in the validation output directory
      3. Verify that your IoU threshold (0.5) matches what's being used for metric calculation

      The metrics implementation computes precision across a range of confidence thresholds and may interpolate values between operating points, which explains why the reported value differs from your expected calculation. For deeper investigation, check the full metrics implementation in the ultralytics/utils/metrics.py file.

    6. maomao123-gitGitHub

      glenn-jocher Apr 18, 2025

      @glenn-jocher, if we can get the ap50-95, I think we also can get recall50-95, precision50-95, f1-score50-95. When I print metrics.box.r(), metrics.box.p()……, I just see the single value of each class. Are the values under IoU = 0.5 if IoU is in default? Is it possible to get recall50-95, precision50-95? thank you!

  5. Fatima-khanjarGitHub

    Hello,

    I hope this message finds you well. I have two questions:

    • There are plots showing different metrics. How can I use them to extract the performance values of the model, given that there are various confidence thresholds? During validation, the default IoU in YOLOv11 is 0.6. Does it remain stable when calculating precision, recall, and F1 score?
    • During validation, I see outputs like this: Class Images Instances Box(P) R mAP50 mAP50-95 all 30 240 0.917 0.911 0.973 0.817 Can I use these values to report the performance of the model? If yes, how can I extract them through coding?

    best regards,

    1. Glenn JocherUltralytics(已编辑)

      For accessing validation metrics programmatically, you can retrieve them directly from the results object after running model.val(). Here's a concise example:

      results = model.val()
      print(f"Precision: {results.box.mp}, Recall: {results.box.mr}, mAP50: {results.box.map50}, mAP50-95: {results.box.map}")

      The default iou=0.7 in val() is the NMS threshold, not the precision/recall threshold. Precision and recall are computed at IoU 0.5 at the confidence that maximizes F1, while mAP50-95 averages over IoU 0.5-0.95. The class-wise table values are valid for performance reporting - full implementation details are in the Model Evaluation Insights guide.

  6. e-roeGitHub

    Hello,

    Looking at the confusion matrices, I find the normalized one strange when compared to the non-normalized version. I decided to train a model with only one class to try to understand it, but the result left me even more confused. Could you explain the meaning of this normalized confusion matrix? Ultralitics version: 8.3.49 Non-normalized: 18953 ---- 159 12 ---- 0

    Normalized: 1.00 ---- 1.00 0.00 ---- 0.00

    Ps. I don't know how to attach an image here Thank you.

    1. Glenn JocherUltralytics(已编辑)

      @e-roe the normalized confusion matrix divides each column (a true class) by its total, so every column sums to 1. Rows are predictions and columns are true classes, including a background row/column. In your single-class case:

      • True object column: 18953 correct detections and 12 missed objects, so 18953/18965 ≈ 1.00 and 12/18965 ≈ 0.00
      • True background column: 159 false positives (background predicted as an object), so 159/159 = 1.00 and 0.00

      For deeper analysis, see our Performance Metrics Guide. Let us know if you need clarification on specific values! 🚀

    2. e-roeGitHub

      @glenn-jocher Thank you for your response.

      If it is row-wise normalization, shouldn't the sum of each row be 1.0? In Scikit-learn, the normalized confusion matrix appears as I expected, in my single-class case: 0.99168062 ---- 0.00831938

      1. ---- 0.

      This result is much easier for me to compare with the non-normalized confusion matrix and interpret. []'s

    3. e-roeGitHub

      Hi, Oh, I see... I think you calculate it this way: cm_normalized = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis] but I was expecting it to be like this: cm_normalized = cm.astype('float') / cm.sum(axis=0)[np.newaxis, :]

      Ok, thank you again.

    4. Glenn JocherUltralytics(已编辑)

      Thank you for catching this nuance! Ultralytics normalizes each column of the confusion matrix, which corresponds to a true class (rows are predictions), rather than each row as in scikit-learn's default. Each column therefore shows how the instances of one true class were predicted.

  7. Keletom1GitHub

    Problem with figuring out the IoU

    Hey there,

    i had been having some trouble with my models and finding out how they perform on new Data. I will explain, what i have been doing and hopefully someone can find out where i was going wrong :D

    What have I done so far:

    I have trained multiple models on different datasets and now i want to find out, how they perform when they are exposed to new data, some of which they have seen, some of which are completely new. The models are all single class segmentation YOLOv8 models and trained for 300 epochs, nothing too special because i just want to see how different models (trained of different datasets) perform on my test dataset. I had all my models run over said test dataset (Prüfdatensatz_2) of 62 images. I had the annotations saved as png and also as JSON. After that i went ahead and annotated all of my 62 images manually with roboflow and exported these as a JSON as well. So now i have the prediction-json files (one for each image) and one big GroundTruth-json file and i want to compare them and get a big results.csv in which the images names, the models they belong to and the Intersection over Union for each image are displayed. I have tried different approaches and the script itself does work, only the IoU-calculation does not seem to work. It's either 0.0 or 1.0 (the latter in case there is nothing to annotate and the model didn't predict anything in this image).

    My Code:
    import json
    import os
    import numpy as np
    import pandas as pd
    import re
    from shapely.geometry import Polygon
    
    # **1. COCO JSON laden**
    coco_json_path = r"C:\Users\wavet\Desktop\OneDrive\Uni\Semester 9\Bachelorarbeit\Datensätze\COCO_Prüfdatensatz_2-2\train\_annotations.coco.json"
    with open(coco_json_path, "r") as f:
        coco_data = json.load(f)
    
    # **2. Mapping-Tabelle für COCO JSON erstellen (Dateiendungen und Suffixe entfernen)**
    def clean_filename(filename):
        cleaned = re.sub(r"_png\.rf\..*|_jpg\.rf\..*", "", filename)  # Entfernt '.rf.<hash>' und Dateiendungen
        print(f"Original: {filename}, Bereinigt: {cleaned}")  # Debug-Ausgabe
        return cleaned
    
    image_id_to_filename = {img["id"]: clean_filename(img["file_name"]) for img in coco_data["images"]}
    gt_annotations = {}
    
    # **3. Konvertiert Segmentierungen in Polygone für IoU-Berechnung**
    def segmentation_to_polygon(segmentation):
        coordinates = [(segmentation[i], segmentation[i+1]) for i in range(0, len(segmentation), 2)]
        if len(coordinates) < 4:
            return None  # Weniger als 4 Punkte = kein gültiges Polygon
        return Polygon(coordinates)
    
    # **4. Ground-Truth-Annotationen nach `image_id` speichern**
    annotations_by_image = {}
    for ann in coco_data["annotations"]:
        img_id = ann["image_id"]
        if img_id not in annotations_by_image:
            annotations_by_image[img_id] = []
        if "segmentation" in ann and len(ann["segmentation"]) > 0:
            polygon = segmentation_to_polygon(ann["segmentation"][0])
            if polygon:
                annotations_by_image[img_id].append(polygon)
    
    # **5. Modell-Outputs einlesen & IoU berechnen**
    predictions_dir = r"C:\Users\wavet\Desktop\OneDrive\Uni\Semester 9\Bachelorarbeit\Ergebnisse_BA\Datensatzstudie\e1000\output_Prüfdatensatz_2_Skaliert"
    csv_output_path = r"C:\Users\wavet\Desktop\OneDrive\Uni\Semester 9\Bachelorarbeit\Ergebnisse_BA\Datensatzstudie\Auswertung_e1000\iou_results_v5.csv"
    
    def calculate_iou(polyA, polyB):
        intersection = polyA.intersection(polyB).area
        union = polyA.union(polyB).area
        iou = intersection / union if union > 0 else 0
        print(f"Intersection: {intersection}, Union: {union}, IoU: {iou}")  # Debug-Ausgabe
        return iou
    
    iou_results = []
    
    for model_name in os.listdir(predictions_dir):
        model_path = os.path.join(predictions_dir, model_name)
        if not os.path.isdir(model_path):
            continue
    
        print(f"Verarbeite Modell: {model_name}")
    
        for json_file in os.listdir(model_path):
            if json_file.endswith(".json"):
                img_name = clean_filename(json_file.replace(".json", ""))
    
                # **6. Passende `image_id` suchen**
                matching_ids = [img_id for img_id, file in image_id_to_filename.items() if file == img_name]
                if not matching_ids:
                    print(f"WARNUNG: Keine `image_id` für {img_name} gefunden!")
                    continue
                image_id = matching_ids[0]  # Falls mehrere IDs vorhanden sind, nehmen wir die erste
    
                # **7. GT-Annotationen laden**
                gt_polygons = annotations_by_image.get(image_id, [])
    
                # **8. Predictions laden**
                with open(os.path.join(model_path, json_file), "r") as f:
                    pred_data = json.load(f)
    
                if not isinstance(pred_data, list):
                    pred_data = [pred_data]  # Falls nur eine Vorhersage vorliegt, in Liste umwandeln
    
                pred_polygons = []
                for pred in pred_data:
                    if "segmentation" in pred and len(pred["segmentation"]) > 0:
                        polygon = segmentation_to_polygon(pred["segmentation"][0])
                        if polygon:
                            pred_polygons.append(polygon)
    
                # **9. IoU berechnen**
                if not gt_polygons and not pred_polygons:
                    avg_iou = 1.0  # Keine GT und keine Prediction → Perfekte Erkennung
                elif not gt_polygons or not pred_polygons:
                    avg_iou = 0.0  # Nur eine der beiden vorhanden → Fehlerkennung
                else:
                    total_iou = 0.0
                    matches = 0
                    used_preds = set()
    
                    for gt_polygon in gt_polygons:
                        best_iou = 0.0
                        best_pred = None
    
                        for i, pred_polygon in enumerate(pred_polygons):
                            if i in used_preds:
                                continue
                            iou = calculate_iou(gt_polygon, pred_polygon)
                            if iou > best_iou:
                                best_iou = iou
                                best_pred = i
    
                        if best_pred is not None:
                            used_preds.add(best_pred)
                            total_iou += best_iou
                            matches += 1
    
                    missing_objects = len(gt_polygons) - matches
                    avg_iou = (total_iou + missing_objects * 0.0) / len(gt_polygons)
    
                iou_results.append([model_name, img_name, avg_iou, np.mean([pred.get("confidence", 100) for pred in pred_data])])
    
    # **10. Ergebnisse speichern**
    iou_df = pd.DataFrame(iou_results, columns=["Model", "Image", "IoU", "Confidence"])
    iou_df.to_csv(csv_output_path, index=False)
    print(f"✅ IoU-Ergebnisse gespeichert unter: {csv_output_path}")
    What i checked:
    • segmentation coordinates are absolute for both JSONs
    • the coordinates seem to match, i was able to plot some of the prediction segmentations
    • alle img_ids were assigned correctly
    Questions:
    • What have I been doing wrong?
    • I have the feeling that there might be a much simpler way to find out how my models perform on new data and I just haven't found it yet. Is that so?

    Thank you all in advance!!

    Greetings, Tom

    1. Glenn JocherUltralytics

      @Keletom1 hi Tom! For evaluating segmentation models, we recommend using YOLOv8's native validation mode which automatically calculates all metrics including mAP and IoU. Simply run:

      from ultralytics import YOLO
      model = YOLO("your-model.pt")
      metrics = model.val(data='your-dataset.yaml')

      This handles all ground-truth/prediction matching and metric calculations out-of-the-box. The validation docs show how to interpret results: https://docs.ultralytics.com/modes/val/. For custom analysis, you could extend the built-in metrics calculation from our AP implementation: https://docs.ultralytics.com/reference/utils/metrics/#ap_per_class

    2. Keletom1GitHub

      Hi Glenn,

      thanks (again) for your help!

      That is some great information and saves quite a lot of work for me. I shoud have discovered that earlier.

      Anyway i still have some minor questions regarding the validation.

      1. The standard output for my segmentation model are plots of precision, recall, precision/recall and F1-score (for both the box and mask predictions). Since i don't need to compare the box recognition, is there any way i can disable this output?
      2. The IoU is not a metric that is calculated, is it? At least it is not in the predictions.json or the plots. Any idea on how i can display or calculate it anyway? Or would you recommend to just work with precision and recall?
      3. Could you briefly explain what exactly the JSON displays. I was expecting coordinates under "segmentation" but it just looks like a bunch of random letters and numbers....

      Here is an example of the first to images in my dataset:

      {"image_id": "-0008_1_png.rf.53cbd30c302485bebd6f33083f141f1c", "category_id": 1, "bbox": [222.615, 9.455, 270.527, 614.687], "score": 0.87661, "segmentation": {"size": [640, 640], "counts": "Uh[43Uch01dPXO3P]OJRb0:n]OMha06X^OM`a08`^OLXa08h^OLPa08P_OLg`09Y_OIb`0:^_OI\\`0:d_OKS`09m_OLj?8X@Ib?:b@FV?7TAJc>7dAHU>:QBFi=:]BE^=9kBHk<9[CJ\\<8iCIP<7WDJa;5gDLo:2ZE0\\:3jELQ:4VFKd93eFOS9OSG5c8MbG3X8NmG3l7K]H7X7IPI8f6L^I4]6MiI2Q6LYJ4`5JfJ<o4EVK;c4GcK9U4HRL;b3ChL>P3DTM=f2DaM:Z2EmM=j1A]Nd0Z1\\OjNe0Q1[OUOd0g0XOCj04TO3n0lKkM@W1h4n0eKoM]OT1U5k0[KRN]OR1`5l0mJSNBn0g5X1\\JjMKm0n5Z1SJkMLl0V6X1kIlMNh0a6^1ZIjM4d0i6h1kHdM:e0o6g1dHdM<d0W7e1[HgM<c0a7g1mGfMa0`0g7S2^G]Mj0?l7U2WG\\Ml0>T8U2lF]Mn0<_8W2]F^MR19g8Z2TF]MT18l8[2mE]MV18U9X2`EbMZ14^9Y2REdM`10c9^2iDbMc10h9]2dDcMc10P:Z2YDgMf1LY:]2mCfMj1I`:b2bCfMm1Hd:b2]CfMn1Hm:^2QCkMm?W2j_OmMT2^Od;`3ZDbLh;\\3nC^L]N2Q>W3aClLl<P3QCSMP=l2iB\\MY=a2dB[Ml=\\2nAZMhN:a?W2fAmM\\>R2cAoM`>n1^AfMnN2U`0j1j@UNe?j1X@WNk?g1U@YNn?d1a1G3M1O2N5K4L2N1O1O000O011O0O10002M2O1O1OO0100O010O10O0010O0010O01O0010O01O001O010O010O1O010O001O10O01O010O100O00100O00100O00100O010O10O0100O00100O010O010O10O01O010O1O010O010O1O010O0010O010O010O01O010O001O010O010O01O0100O00010O010O010O0010O01O01O010O010O01O01O01O000010O00010O01O1O001O00_kj2"}}, {"image_id": "-0008_1_png.rf.53cbd30c302485bebd6f33083f141f1c", "category_id": 1, "bbox": [148.255, 393.207, 71.421, 246.348], "score": 0.00426, "segmentation": {"size": [640, 640], "counts": "Wgm2b0\\c0O1O00O2OO01O010100OO0O21O00N01200OO0O111O0OO2O1100N00010000N12O10OO1N030O00OO11100OM20101OO1M311O0N00O21O00OO21O10OOO1110OOO110100N1O01101NhSV8"}}, {"image_id": "-0008_1_png.rf.53cbd30c302485bebd6f33083f141f1c", "category_id": 1, "bbox": [146.245, 394.098, 186.875, 244.714], "score": 0.00358, "segmentation": {"size": [640, 640], "counts": "`_l2?\\c02O11O1O00O2OO01O010100OO0O21O0OO1020O0O00011O0OO2O1100N0001000OO21O10OO1N030O00OO11100ON1O2010N0N311O0N00O21O00OO21O10OOO1110OO0001100N1O01101N3I7N2N[og02_PXO>F2K5L4L4K5M3K5M1ON11N30OOM30N4OO1N11M3010J40N4N10M4N12M11M1OO4N2ON3O01O1NOO112N20N2N2O101L4M21O0OO3M3O1O0M3N3O001N1N30O00M3N30O01M2N21O10M2M40mjn5"}}

      Thank you very much!

      Kind regards, Tom

    3. Glenn JocherUltralytics

      Hi Tom,

      1. Disabling box metrics: Currently, segmentation validation outputs both box and mask metrics together. You can safely ignore the box-related metrics/plots if you're focused on segmentation, but there's no built-in flag to disable box outputs specifically.

      2. IoU calculation: While per-image IoU isn't directly output, mAP metrics (mAP50, mAP50-95) in validation results are calculated using IoU thresholds. For custom per-instance IoU calculations, see our mask_iou function in the metrics utils.

      3. Segmentation format: The JSON uses Run-Length Encoding (RLE) for compact storage of masks. You can decode these with pycocotools (COCO API) or use our native Results class to access polygon coordinates directly from predictions.

      For deeper analysis, consider using the COCO evaluation tools which handle these formats natively.

      1
  8. vonzoGitHub

    After training I get the weights for "last" and "best", how do i know which line in the results.csv correspond to the "best" weights?

    1. Glenn JocherUltralytics(已编辑)

      @vonzo the 'best.pt' weights correspond to the epoch with the highest validation mAP50-95 metric. In your results.csv, look for the row where the metrics/mAP50-95(B) column (or similar mAP column for your task) reaches its maximum value - this is the epoch where best.pt was last saved.

    1. Glenn JocherUltralytics

      @Aniket1210 to measure latency for the YOLOv8 TSR model on an RTX 4090, you can use the model.val() or model.predict() functions with the verbose=True parameter in the Ultralytics API to retrieve timing details. Ensure your setup leverages the GPU (device='0') for accurate benchmarking. For further guidance, refer to the Ultralytics Val documentation.

  9. xiaoshuai123666GitHub

    这是来自QQ邮箱的自动回复邮件。你好,您的邮件我已收到,感谢您的来信,我一定仔细阅读您的邮件。                                                            ——xiaoshuai