Reference for ultralytics/utils/benchmarks.py
Note
This file is available at https://github.com/ultralytics/ultralytics/blob/main/ultralytics/utils/benchmarks.py. If you spot a problem please help fix it by contributing a Pull Request 🛠️. Thank you 🙏!
ultralytics.utils.benchmarks.RF100Benchmark
Benchmark YOLO model performance across various formats for speed and accuracy.
Source code in ultralytics/utils/benchmarks.py
evaluate
Evaluate model performance on validation results.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
yaml_path
|
str
|
Path to the YAML configuration file. |
required |
val_log_file
|
str
|
Path to the validation log file. |
required |
eval_log_file
|
str
|
Path to the evaluation log file. |
required |
list_ind
|
int
|
Index of the current dataset in the list. |
required |
Returns:
Type | Description |
---|---|
float
|
The mean average precision (mAP) value for the evaluated model. |
Examples:
Evaluate a model on a specific dataset
>>> benchmark = RF100Benchmark()
>>> benchmark.evaluate("path/to/data.yaml", "path/to/val_log.txt", "path/to/eval_log.txt", 0)
Source code in ultralytics/utils/benchmarks.py
fix_yaml
staticmethod
Fixes the train and validation paths in a given YAML file.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path
|
str
|
Path to the YAML file to be fixed. |
required |
Examples:
Source code in ultralytics/utils/benchmarks.py
parse_dataset
Parse dataset links and download datasets.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
ds_link_txt
|
str
|
Path to the file containing dataset links. |
'datasets_links.txt'
|
Examples:
>>> benchmark = RF100Benchmark()
>>> benchmark.set_key("api_key")
>>> benchmark.parse_dataset("datasets_links.txt")
Source code in ultralytics/utils/benchmarks.py
set_key
Set Roboflow API key for processing.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
api_key
|
str
|
The API key. |
required |
Examples:
Set the Roboflow API key for accessing datasets:
Source code in ultralytics/utils/benchmarks.py
ultralytics.utils.benchmarks.ProfileModels
ProfileModels(
paths: list,
num_timed_runs=100,
num_warmup_runs=10,
min_time=60,
imgsz=640,
half=True,
trt=True,
device=None,
)
ProfileModels class for profiling different models on ONNX and TensorRT.
This class profiles the performance of different models, returning results such as model speed and FLOPs.
Attributes:
Name | Type | Description |
---|---|---|
paths |
List[str]
|
Paths of the models to profile. |
num_timed_runs |
int
|
Number of timed runs for the profiling. |
num_warmup_runs |
int
|
Number of warmup runs before profiling. |
min_time |
float
|
Minimum number of seconds to profile for. |
imgsz |
int
|
Image size used in the models. |
half |
bool
|
Flag to indicate whether to use FP16 half-precision for TensorRT profiling. |
trt |
bool
|
Flag to indicate whether to profile using TensorRT. |
device |
device
|
Device used for profiling. |
Methods:
Name | Description |
---|---|
profile |
Profiles the models and prints the result. |
Examples:
Profile models and print results
>>> from ultralytics.utils.benchmarks import ProfileModels
>>> profiler = ProfileModels(["yolov8n.yaml", "yolov8s.yaml"], imgsz=640)
>>> profiler.profile()
Parameters:
Name | Type | Description | Default |
---|---|---|---|
paths
|
List[str]
|
List of paths of the models to be profiled. |
required |
num_timed_runs
|
int
|
Number of timed runs for the profiling. |
100
|
num_warmup_runs
|
int
|
Number of warmup runs before the actual profiling starts. |
10
|
min_time
|
float
|
Minimum time in seconds for profiling a model. |
60
|
imgsz
|
int
|
Size of the image used during profiling. |
640
|
half
|
bool
|
Flag to indicate whether to use FP16 half-precision for TensorRT profiling. |
True
|
trt
|
bool
|
Flag to indicate whether to profile using TensorRT. |
True
|
device
|
device | None
|
Device used for profiling. If None, it is determined automatically. |
None
|
Notes
FP16 'half' argument option removed for ONNX as slower on CPU than FP32.
Examples:
Initialize and profile models
>>> from ultralytics.utils.benchmarks import ProfileModels
>>> profiler = ProfileModels(["yolov8n.yaml", "yolov8s.yaml"], imgsz=640)
>>> profiler.profile()
Source code in ultralytics/utils/benchmarks.py
generate_results_dict
staticmethod
Generates a dictionary of profiling results including model name, parameters, GFLOPs, and speed metrics.
Source code in ultralytics/utils/benchmarks.py
generate_table_row
Generates a table row string with model performance metrics including inference times and model details.
Source code in ultralytics/utils/benchmarks.py
get_files
Returns a list of paths for all relevant model files given by the user.
Source code in ultralytics/utils/benchmarks.py
get_onnx_model_info
Extracts metadata from an ONNX model file including parameters, GFLOPs, and input shape.
iterative_sigma_clipping
staticmethod
Applies iterative sigma clipping to data to remove outliers based on specified sigma and iteration count.
Source code in ultralytics/utils/benchmarks.py
print_table
staticmethod
Prints a formatted table of model profiling results, including speed and accuracy metrics.
Source code in ultralytics/utils/benchmarks.py
profile
Profiles YOLO models for speed and accuracy across various formats including ONNX and TensorRT.
Source code in ultralytics/utils/benchmarks.py
profile_onnx_model
Profiles an ONNX model, measuring average inference time and standard deviation across multiple runs.
Source code in ultralytics/utils/benchmarks.py
profile_tensorrt_model
Profiles YOLO model performance with TensorRT, measuring average run time and standard deviation.
Source code in ultralytics/utils/benchmarks.py
ultralytics.utils.benchmarks.benchmark
benchmark(model=WEIGHTS_DIR / 'yolo11n.pt', data=None, imgsz=160, half=False, int8=False, device='cpu', verbose=False, eps=0.001)
Benchmark a YOLO model across different formats for speed and accuracy.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model
|
str | Path
|
Path to the model file or directory. |
WEIGHTS_DIR / 'yolo11n.pt'
|
data
|
str | None
|
Dataset to evaluate on, inherited from TASK2DATA if not passed. |
None
|
imgsz
|
int
|
Image size for the benchmark. |
160
|
half
|
bool
|
Use half-precision for the model if True. |
False
|
int8
|
bool
|
Use int8-precision for the model if True. |
False
|
device
|
str
|
Device to run the benchmark on, either 'cpu' or 'cuda'. |
'cpu'
|
verbose
|
bool | float
|
If True or a float, assert benchmarks pass with given metric. |
False
|
eps
|
float
|
Epsilon value for divide by zero prevention. |
0.001
|
Returns:
Type | Description |
---|---|
DataFrame
|
A pandas DataFrame with benchmark results for each format, including file size, metric, and inference time. |
Examples:
Benchmark a YOLO model with default settings:
Source code in ultralytics/utils/benchmarks.py
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 |
|