شرح مفصل لمقاييس الأداء#

@atharvahude output of mAP₅₀ for a single class in fact refer to AP₅₀. You can also refer to the following discussion on the same. https://github.com/orgs/ultralytics/discussions/8790#discussioncomment-9476915

@atharvahude the per-class map50 and map50-95 values should indeed be treated as AP for those specific classes. The "all" row represents the mean AP across all classes, confirming your understanding. For more details, you can refer to our documentation on performance metrics.

@e-roe great observation! You're right that the training loss plots could use more explanation. The training losses shown during YOLO11 training represent:

• train/box_loss: Localization loss measuring how well predicted bounding boxes match ground truth coordinates
• train/cls_loss: Classification loss measuring how accurately the model predicts object classes
• train/dfl_loss: Distribution Focal Loss, which helps improve localization accuracy by treating bounding box regression as a classification problem (YOLO26 is DFL-free, so this term is a plain L1 box-distance loss there)

These training losses should generally decrease over epochs, indicating the model is learning. The validation losses (val/box_loss, val/cls_loss, val/dfl_loss) help monitor overfitting - if training losses decrease but validation losses increase or plateau, it suggests overfitting.

Thanks for pointing this out! We'll consider expanding the training loss explanations in future documentation updates.

@Fatima-khanjar excellent questions about model evaluation!

For DFL, classification, and box losses, there are no absolute "good" values as they depend on your dataset complexity, model size, and task. What's more important is the trend - consistently decreasing losses indicate proper training. Loss values of 0.3-0.6 with high precision/recall (>0.8) can be completely normal, as metrics often continue improving even after losses plateau.

To extract training/validation loss values programmatically:

import pandas as pd

# Load the results CSV
results_df = pd.read_csv('path/to/results.csv')

# Extract train/val losses
train_box_loss = results_df['train/box_loss']
val_box_loss = results_df['val/box_loss']
train_cls_loss = results_df['train/cls_loss']
val_cls_loss = results_df['val/cls_loss']
train_dfl_loss = results_df['train/dfl_loss']
val_dfl_loss = results_df['val/dfl_loss']

Focus on comparing validation metrics against your application requirements rather than trying to reach arbitrary loss values. A model with slightly higher losses but better real-world performance is ultimately more valuable.

Hi！how to print f1 score in model.val() ? In addition, I want to print metrics with four decimal places so that I can get percentage form. For example, the printed mAP₅₀ was 0.8657, and I can transfer the number to 86.57%. Where and how can I revise the code? Thanks!

@maomao123-git to print the F1 score and format metrics with four decimal places when running validation, you can access these values from the results object:

from ultralytics import YOLO

# Load your model
model = YOLO('yolo26n.pt')  # or your custom model path

# Run validation
results = model.val()

# results.box.f1 holds one F1 score per class
f1 = results.box.f1.mean()
map50 = results.box.map50

# Format and print
print(f"F1 Score: {f1:.4f} ({f1*100:.2f}%)")
print(f"mAP50: {map50:.4f} ({map50*100:.2f}%)")

# For class-specific metrics
for i, f1_val in enumerate(results.box.f1):
    class_name = model.names[results.box.ap_class_index[i]]
    print(f"Class '{class_name}' F1 Score: {f1_val:.4f} ({f1_val*100:.2f}%)")

This gives you access to both overall and class-specific F1 scores with your desired formatting. The metrics class includes all the performance metrics like precision, recall, mAP, and F1 score, which you can format as needed.

Thanks for your help. I can print f1 of per class, yet there is no f1 of class "all". How can I get it?

@maomao123-git to get the F1 score for "all" classes (mean F1 across all classes), you can calculate it from the overall precision and recall values:

# Run validation
results = model.val()

# Get overall precision and recall
precision_all = results.box.mp  # mean precision
recall_all = results.box.mr     # mean recall

# Calculate F1 score for "all" classes
f1_all = 2 * (precision_all * recall_all) / (precision_all + recall_all) if (precision_all + recall_all) > 0 else 0

# Format and print
print(f"All Classes F1 Score: {f1_all:.4f} ({f1_all*100:.2f}%)")

Alternatively, you can calculate the mean of class-specific F1 scores:

# Calculate mean of all class F1 scores
mean_f1 = results.box.f1.mean()
print(f"Mean F1 Score: {mean_f1:.4f} ({mean_f1*100:.2f}%)")

Both methods should give you a good representation of the overall F1 score that would appear in the "all" row.

thanks！

@Shubham77saini the precision calculation in YOLOv8 considers confidence thresholds and precision-recall curve interpolation, which can lead to different values than direct TP/(TP+FP) calculations. The reported 0.734 precision for the "car" class is likely affected by how the model handles confidence scores across the precision-recall curve.

To understand this better, you might want to:

1. Check the confusion matrix at different confidence thresholds
2. Look at P-R curves in the validation output directory
3. Verify that your IoU threshold (0.5) matches what's being used for metric calculation

The metrics implementation computes precision across a range of confidence thresholds and may interpolate values between operating points, which explains why the reported value differs from your expected calculation. For deeper investigation, check the full metrics implementation in the ultralytics/utils/metrics.py file.

glenn-jocher Apr 18, 2025

@glenn-jocher, if we can get the ap50-95, I think we also can get recall50-95, precision50-95, f1-score50-95. When I print metrics.box.r(), metrics.box.p()……, I just see the single value of each class. Are the values under IoU = 0.5 if IoU is in default? Is it possible to get recall50-95, precision50-95? thank you!

For accessing validation metrics programmatically, you can retrieve them directly from the results object after running model.val(). Here's a concise example:

results = model.val()
print(f"Precision: {results.box.mp}, Recall: {results.box.mr}, mAP50: {results.box.map50}, mAP50-95: {results.box.map}")

The default iou=0.7 in val() is the NMS threshold, not the precision/recall threshold. Precision and recall are computed at IoU 0.5 at the confidence that maximizes F1, while mAP_50-95 averages over IoU 0.5-0.95. The class-wise table values are valid for performance reporting - full implementation details are in the Model Evaluation Insights guide.

@e-roe the normalized confusion matrix divides each column (a true class) by its total, so every column sums to 1. Rows are predictions and columns are true classes, including a background row/column. In your single-class case:

• True object column: 18953 correct detections and 12 missed objects, so 18953/18965 ≈ 1.00 and 12/18965 ≈ 0.00
• True background column: 159 false positives (background predicted as an object), so 159/159 = 1.00 and 0.00

For deeper analysis, see our Performance Metrics Guide. Let us know if you need clarification on specific values! 🚀

@glenn-jocher Thank you for your response.

If it is row-wise normalization, shouldn't the sum of each row be 1.0? In Scikit-learn, the normalized confusion matrix appears as I expected, in my single-class case: 0.99168062 ---- 0.00831938

1. ---- 0.

This result is much easier for me to compare with the non-normalized confusion matrix and interpret. []'s

Hi, Oh, I see... I think you calculate it this way: cm_normalized = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis] but I was expecting it to be like this: cm_normalized = cm.astype('float') / cm.sum(axis=0)[np.newaxis, :]

Ok, thank you again.

Thank you for catching this nuance! Ultralytics normalizes each column of the confusion matrix, which corresponds to a true class (rows are predictions), rather than each row as in scikit-learn's default. Each column therefore shows how the instances of one true class were predicted.

@Keletom1 hi Tom! For evaluating segmentation models, we recommend using YOLOv8's native validation mode which automatically calculates all metrics including mAP and IoU. Simply run:

from ultralytics import YOLO
model = YOLO("your-model.pt")
metrics = model.val(data='your-dataset.yaml')

This handles all ground-truth/prediction matching and metric calculations out-of-the-box. The validation docs show how to interpret results: https://docs.ultralytics.com/modes/val/. For custom analysis, you could extend the built-in metrics calculation from our AP implementation: https://docs.ultralytics.com/reference/utils/metrics/#ap_per_class

Hi Glenn,

thanks (again) for your help!

That is some great information and saves quite a lot of work for me. I shoud have discovered that earlier.

Anyway i still have some minor questions regarding the validation.

1. The standard output for my segmentation model are plots of precision, recall, precision/recall and F1-score (for both the box and mask predictions). Since i don't need to compare the box recognition, is there any way i can disable this output?
2. The IoU is not a metric that is calculated, is it? At least it is not in the predictions.json or the plots. Any idea on how i can display or calculate it anyway? Or would you recommend to just work with precision and recall?
3. Could you briefly explain what exactly the JSON displays. I was expecting coordinates under "segmentation" but it just looks like a bunch of random letters and numbers....

Here is an example of the first to images in my dataset:

{"image_id": "-0008_1_png.rf.53cbd30c302485bebd6f33083f141f1c", "category_id": 1, "bbox": [222.615, 9.455, 270.527, 614.687], "score": 0.87661, "segmentation": {"size": [640, 640], "counts": "Uh[43Uch01dPXO3P]OJRb0:n]OMha06X^OM`a08`^OLXa08h^OLPa08P_OLg`09Y_OIb`0:^_OI\\`0:d_OKS`09m_OLj?8X@Ib?:b@FV?7TAJc>7dAHU>:QBFi=:]BE^=9kBHk<9[CJ\\<8iCIP<7WDJa;5gDLo:2ZE0\\:3jELQ:4VFKd93eFOS9OSG5c8MbG3X8NmG3l7K]H7X7IPI8f6L^I4]6MiI2Q6LYJ4`5JfJ<o4EVK;c4GcK9U4HRL;b3ChL>P3DTM=f2DaM:Z2EmM=j1A]Nd0Z1\\OjNe0Q1[OUOd0g0XOCj04TO3n0lKkM@W1h4n0eKoM]OT1U5k0[KRN]OR1`5l0mJSNBn0g5X1\\JjMKm0n5Z1SJkMLl0V6X1kIlMNh0a6^1ZIjM4d0i6h1kHdM:e0o6g1dHdM<d0W7e1[HgM<c0a7g1mGfMa0`0g7S2^G]Mj0?l7U2WG\\Ml0>T8U2lF]Mn0<_8W2]F^MR19g8Z2TF]MT18l8[2mE]MV18U9X2`EbMZ14^9Y2REdM`10c9^2iDbMc10h9]2dDcMc10P:Z2YDgMf1LY:]2mCfMj1I`:b2bCfMm1Hd:b2]CfMn1Hm:^2QCkMm?W2j_OmMT2^Od;`3ZDbLh;\\3nC^L]N2Q>W3aClLl<P3QCSMP=l2iB\\MY=a2dB[Ml=\\2nAZMhN:a?W2fAmM\\>R2cAoM`>n1^AfMnN2U`0j1j@UNe?j1X@WNk?g1U@YNn?d1a1G3M1O2N5K4L2N1O1O000O011O0O10002M2O1O1OO0100O010O10O0010O0010O01O0010O01O001O010O010O1O010O001O10O01O010O100O00100O00100O00100O010O10O0100O00100O010O010O10O01O010O1O010O010O1O010O0010O010O010O01O010O001O010O010O01O0100O00010O010O010O0010O01O01O010O010O01O01O01O000010O00010O01O1O001O00_kj2"}}, {"image_id": "-0008_1_png.rf.53cbd30c302485bebd6f33083f141f1c", "category_id": 1, "bbox": [148.255, 393.207, 71.421, 246.348], "score": 0.00426, "segmentation": {"size": [640, 640], "counts": "Wgm2b0\\c0O1O00O2OO01O010100OO0O21O00N01200OO0O111O0OO2O1100N00010000N12O10OO1N030O00OO11100OM20101OO1M311O0N00O21O00OO21O10OOO1110OOO110100N1O01101NhSV8"}}, {"image_id": "-0008_1_png.rf.53cbd30c302485bebd6f33083f141f1c", "category_id": 1, "bbox": [146.245, 394.098, 186.875, 244.714], "score": 0.00358, "segmentation": {"size": [640, 640], "counts": "`_l2?\\c02O11O1O00O2OO01O010100OO0O21O0OO1020O0O00011O0OO2O1100N0001000OO21O10OO1N030O00OO11100ON1O2010N0N311O0N00O21O00OO21O10OOO1110OO0001100N1O01101N3I7N2N[og02_PXO>F2K5L4L4K5M3K5M1ON11N30OOM30N4OO1N11M3010J40N4N10M4N12M11M1OO4N2ON3O01O1NOO112N20N2N2O101L4M21O0OO3M3O1O0M3N3O001N1N30O00M3N30O01M2N21O10M2M40mjn5"}}

Thank you very much!

Kind regards, Tom

Hi Tom,

1. Disabling box metrics: Currently, segmentation validation outputs both box and mask metrics together. You can safely ignore the box-related metrics/plots if you're focused on segmentation, but there's no built-in flag to disable box outputs specifically.

2. IoU calculation: While per-image IoU isn't directly output, mAP metrics (mAP₅₀, mAP_50-95) in validation results are calculated using IoU thresholds. For custom per-instance IoU calculations, see our mask_iou function in the metrics utils.

3. Segmentation format: The JSON uses Run-Length Encoding (RLE) for compact storage of masks. You can decode these with pycocotools (COCO API) or use our native Results class to access polygon coordinates directly from predictions.

For deeper analysis, consider using the COCO evaluation tools which handle these formats natively.

@vonzo the 'best.pt' weights correspond to the epoch with the highest validation mAP_50-95 metric. In your results.csv, look for the row where the metrics/mAP50-95(B) column (or similar mAP column for your task) reaches its maximum value - this is the epoch where best.pt was last saved.

@Aniket1210 to measure latency for the YOLOv8 TSR model on an RTX 4090, you can use the model.val() or model.predict() functions with the verbose=True parameter in the Ultralytics API to retrieve timing details. Ensure your setup leverages the GPU (device='0') for accurate benchmarking. For further guidance, refer to the Ultralytics Val documentation.