Reference for ultralytics/utils/benchmarks.py
Note
This file is available at https://github.com/ultralytics/ultralytics/blob/main/ultralytics/utils/benchmarks.py. If you spot a problem please help fix it by contributing a Pull Request 🛠️. Thank you 🙏!
ultralytics.utils.benchmarks.RF100Benchmark
Benchmark YOLO model performance across formats for speed and accuracy.
Source code in ultralytics/utils/benchmarks.py
evaluate
Model evaluation on validation results.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
yaml_path |
str
|
YAML file path. |
required |
val_log_file |
str
|
val_log_file path. |
required |
eval_log_file |
str
|
eval_log_file path. |
required |
list_ind |
int
|
Index for current dataset. |
required |
Source code in ultralytics/utils/benchmarks.py
fix_yaml
staticmethod
Function to fix YAML train and val path.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path |
str
|
YAML file path. |
required |
Source code in ultralytics/utils/benchmarks.py
parse_dataset
Parse dataset links and downloads datasets.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
ds_link_txt |
str
|
Path to dataset_links file. |
'datasets_links.txt'
|
Source code in ultralytics/utils/benchmarks.py
set_key
Set Roboflow API key for processing.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
api_key |
str
|
The API key. |
required |
ultralytics.utils.benchmarks.ProfileModels
ProfileModels(paths: list, num_timed_runs=100, num_warmup_runs=10, min_time=60, imgsz=640, half=True, trt=True, device=None)
ProfileModels class for profiling different models on ONNX and TensorRT.
This class profiles the performance of different models, returning results such as model speed and FLOPs.
Attributes:
Name | Type | Description |
---|---|---|
paths |
list
|
Paths of the models to profile. |
num_timed_runs |
int
|
Number of timed runs for the profiling. Default is 100. |
num_warmup_runs |
int
|
Number of warmup runs before profiling. Default is 10. |
min_time |
float
|
Minimum number of seconds to profile for. Default is 60. |
imgsz |
int
|
Image size used in the models. Default is 640. |
Methods:
Name | Description |
---|---|
profile |
Profiles the models and prints the result. |
Example
Parameters:
Name | Type | Description | Default |
---|---|---|---|
paths |
list
|
List of paths of the models to be profiled. |
required |
num_timed_runs |
int
|
Number of timed runs for the profiling. Default is 100. |
100
|
num_warmup_runs |
int
|
Number of warmup runs before the actual profiling starts. Default is 10. |
10
|
min_time |
float
|
Minimum time in seconds for profiling a model. Default is 60. |
60
|
imgsz |
int
|
Size of the image used during profiling. Default is 640. |
640
|
half |
bool
|
Flag to indicate whether to use half-precision floating point for profiling. |
True
|
trt |
bool
|
Flag to indicate whether to profile using TensorRT. Default is True. |
True
|
device |
device
|
Device used for profiling. If None, it is determined automatically. |
None
|
Source code in ultralytics/utils/benchmarks.py
generate_results_dict
staticmethod
Generates a dictionary of model details including name, parameters, GFLOPS and speed metrics.
Source code in ultralytics/utils/benchmarks.py
generate_table_row
Generates a formatted string for a table row that includes model performance and metric details.
Source code in ultralytics/utils/benchmarks.py
get_files
Returns a list of paths for all relevant model files given by the user.
Source code in ultralytics/utils/benchmarks.py
get_onnx_model_info
Retrieves the information including number of layers, parameters, gradients and FLOPs for an ONNX model file.
Source code in ultralytics/utils/benchmarks.py
iterative_sigma_clipping
staticmethod
Applies an iterative sigma clipping algorithm to the given data times number of iterations.
Source code in ultralytics/utils/benchmarks.py
print_table
staticmethod
Formats and prints a comparison table for different models with given statistics and performance data.
Source code in ultralytics/utils/benchmarks.py
profile
Logs the benchmarking results of a model, checks metrics against floor and returns the results.
Source code in ultralytics/utils/benchmarks.py
profile_onnx_model
Profiles an ONNX model by executing it multiple times and returns the mean and standard deviation of run times.
Source code in ultralytics/utils/benchmarks.py
profile_tensorrt_model
Profiles the TensorRT model, measuring average run time and standard deviation among runs.
Source code in ultralytics/utils/benchmarks.py
ultralytics.utils.benchmarks.benchmark
benchmark(model=WEIGHTS_DIR / 'yolov8n.pt', data=None, imgsz=160, half=False, int8=False, device='cpu', verbose=False)
Benchmark a YOLO model across different formats for speed and accuracy.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model |
str | Path | optional
|
Path to the model file or directory. Default is Path(SETTINGS['weights_dir']) / 'yolov8n.pt'. |
WEIGHTS_DIR / 'yolov8n.pt'
|
data |
str
|
Dataset to evaluate on, inherited from TASK2DATA if not passed. Default is None. |
None
|
imgsz |
int
|
Image size for the benchmark. Default is 160. |
160
|
half |
bool
|
Use half-precision for the model if True. Default is False. |
False
|
int8 |
bool
|
Use int8-precision for the model if True. Default is False. |
False
|
device |
str
|
Device to run the benchmark on, either 'cpu' or 'cuda'. Default is 'cpu'. |
'cpu'
|
verbose |
bool | float | optional
|
If True or a float, assert benchmarks pass with given metric. Default is False. |
False
|
Returns:
Name | Type | Description |
---|---|---|
df |
DataFrame
|
A pandas DataFrame with benchmark results for each format, including file size, metric, and inference time. |
Source code in ultralytics/utils/benchmarks.py
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 |
|