Reference for ultralytics/nn/modules/utils.py
Note
This file is available at https://github.com/ultralytics/ultralytics/blob/main/ultralytics/nn/modules/utils.py. If you spot a problem please help fix it by contributing a Pull Request 🛠️. Thank you 🙏!
ultralytics.nn.modules.utils._get_clones
Create a list of cloned modules from the given module.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
module
|
Module
|
The module to be cloned. |
required |
n
|
int
|
Number of clones to create. |
required |
Returns:
Type | Description |
---|---|
ModuleList
|
A ModuleList containing n clones of the input module. |
Examples:
>>> import torch.nn as nn
>>> layer = nn.Linear(10, 10)
>>> clones = _get_clones(layer, 3)
>>> len(clones)
3
Source code in ultralytics/nn/modules/utils.py
ultralytics.nn.modules.utils.bias_init_with_prob
Initialize conv/fc bias value according to a given probability value.
This function calculates the bias initialization value based on a prior probability using the inverse error function. It's commonly used in object detection models to initialize classification layers with a specific positive prediction probability.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prior_prob
|
float
|
Prior probability for bias initialization. |
0.01
|
Returns:
Type | Description |
---|---|
float
|
Bias initialization value calculated from the prior probability. |
Examples:
>>> bias = bias_init_with_prob(0.01)
>>> print(f"Bias initialization value: {bias:.4f}")
Bias initialization value: -4.5951
Source code in ultralytics/nn/modules/utils.py
ultralytics.nn.modules.utils.linear_init
Initialize the weights and biases of a linear module.
This function initializes the weights of a linear module using a uniform distribution within bounds calculated from the input dimension. If the module has a bias, it is also initialized.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
module
|
Module
|
Linear module to initialize. |
required |
Returns:
Type | Description |
---|---|
Module
|
The initialized module. |
Examples:
>>> import torch.nn as nn
>>> linear = nn.Linear(10, 5)
>>> initialized_linear = linear_init(linear)
Source code in ultralytics/nn/modules/utils.py
ultralytics.nn.modules.utils.inverse_sigmoid
Calculate the inverse sigmoid function for a tensor.
This function applies the inverse of the sigmoid function to a tensor, which is useful in various neural network operations, particularly in attention mechanisms and coordinate transformations.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
x
|
Tensor
|
Input tensor with values in range [0, 1]. |
required |
eps
|
float
|
Small epsilon value to prevent numerical instability. |
1e-05
|
Returns:
Type | Description |
---|---|
Tensor
|
Tensor after applying the inverse sigmoid function. |
Examples:
Source code in ultralytics/nn/modules/utils.py
ultralytics.nn.modules.utils.multi_scale_deformable_attn_pytorch
multi_scale_deformable_attn_pytorch(
value: Tensor,
value_spatial_shapes: Tensor,
sampling_locations: Tensor,
attention_weights: Tensor,
) -> torch.Tensor
Implement multi-scale deformable attention in PyTorch.
This function performs deformable attention across multiple feature map scales, allowing the model to attend to different spatial locations with learned offsets.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
value
|
Tensor
|
The value tensor with shape (bs, num_keys, num_heads, embed_dims). |
required |
value_spatial_shapes
|
Tensor
|
Spatial shapes of the value tensor with shape (num_levels, 2). |
required |
sampling_locations
|
Tensor
|
The sampling locations with shape (bs, num_queries, num_heads, num_levels, num_points, 2). |
required |
attention_weights
|
Tensor
|
The attention weights with shape (bs, num_queries, num_heads, num_levels, num_points). |
required |
Returns:
Type | Description |
---|---|
Tensor
|
The output tensor with shape (bs, num_queries, embed_dims). |
References
https://github.com/IDEA-Research/detrex/blob/main/detrex/layers/multi_scale_deform_attn.py