Reference for ultralytics/nn/modules/transformer.py
Note
This file is available at https://github.com/ultralytics/ultralytics/blob/main/ultralytics/nn/modules/transformer.py. If you spot a problem please help fix it by contributing a Pull Request 🛠️. Thank you 🙏!
ultralytics.nn.modules.transformer.TransformerEncoderLayer
TransformerEncoderLayer(
c1, cm=2048, num_heads=8, dropout=0.0, act=nn.GELU(), normalize_before=False
)
Bases: Module
Defines a single layer of the transformer encoder.
Source code in ultralytics/nn/modules/transformer.py
forward
Forward propagates the input through the encoder module.
Source code in ultralytics/nn/modules/transformer.py
forward_post
Performs forward pass with post-normalization.
Source code in ultralytics/nn/modules/transformer.py
forward_pre
Performs forward pass with pre-normalization.
Source code in ultralytics/nn/modules/transformer.py
with_pos_embed staticmethod
ultralytics.nn.modules.transformer.AIFI
Bases: TransformerEncoderLayer
Defines the AIFI transformer layer.
Source code in ultralytics/nn/modules/transformer.py
build_2d_sincos_position_embedding staticmethod
Builds 2D sine-cosine position embedding.
Source code in ultralytics/nn/modules/transformer.py
forward
Forward pass for the AIFI transformer layer.
Source code in ultralytics/nn/modules/transformer.py
ultralytics.nn.modules.transformer.TransformerLayer
Bases: Module
Transformer layer https://arxiv.org/abs/2010.11929 (LayerNorm layers removed for better performance).
Source code in ultralytics/nn/modules/transformer.py
forward
Apply a transformer block to the input x and return the output.
ultralytics.nn.modules.transformer.TransformerBlock
Bases: Module
Vision Transformer https://arxiv.org/abs/2010.11929.
Source code in ultralytics/nn/modules/transformer.py
forward
Forward propagates the input through the bottleneck module.
Source code in ultralytics/nn/modules/transformer.py
ultralytics.nn.modules.transformer.MLPBlock
Bases: Module
Implements a single block of a multi-layer perceptron.
Source code in ultralytics/nn/modules/transformer.py
ultralytics.nn.modules.transformer.MLP
Bases: Module
Implements a simple multi-layer perceptron (also called FFN).
Source code in ultralytics/nn/modules/transformer.py
forward
Forward pass for the entire MLP.
Source code in ultralytics/nn/modules/transformer.py
ultralytics.nn.modules.transformer.LayerNorm2d
Bases: Module
2D Layer Normalization module inspired by Detectron2 and ConvNeXt implementations.
Original implementations in https://github.com/facebookresearch/detectron2/blob/main/detectron2/layers/batch_norm.py and https://github.com/facebookresearch/ConvNeXt/blob/main/models/convnext.py.
Source code in ultralytics/nn/modules/transformer.py
forward
Perform forward pass for 2D layer normalization.
Source code in ultralytics/nn/modules/transformer.py
ultralytics.nn.modules.transformer.MSDeformAttn
Bases: Module
Multiscale Deformable Attention Module based on Deformable-DETR and PaddleDetection implementations.
https://github.com/fundamentalvision/Deformable-DETR/blob/main/models/ops/modules/ms_deform_attn.py
Source code in ultralytics/nn/modules/transformer.py
forward
Perform forward pass for multiscale deformable attention.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
query | Tensor | [bs, query_length, C] | required |
refer_bbox | Tensor | [bs, query_length, n_levels, 2], range in [0, 1], top-left (0,0), bottom-right (1, 1), including padding area | required |
value | Tensor | [bs, value_length, C] | required |
value_shapes | List | [n_levels, 2], [(H_0, W_0), (H_1, W_1), ..., (H_{L-1}, W_{L-1})] | required |
value_mask | Tensor | [bs, value_length], True for non-padding elements, False for padding elements | None |
Returns:
Name | Type | Description |
---|---|---|
output | Tensor | [bs, Length_{query}, C] |
Source code in ultralytics/nn/modules/transformer.py
ultralytics.nn.modules.transformer.DeformableTransformerDecoderLayer
DeformableTransformerDecoderLayer(
d_model=256,
n_heads=8,
d_ffn=1024,
dropout=0.0,
act=nn.ReLU(),
n_levels=4,
n_points=4,
)
Bases: Module
Deformable Transformer Decoder Layer inspired by PaddleDetection and Deformable-DETR implementations.
https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/transformers/deformable_transformer.py https://github.com/fundamentalvision/Deformable-DETR/blob/main/models/deformable_transformer.py
Source code in ultralytics/nn/modules/transformer.py
forward
Perform the forward pass through the entire decoder layer.
Source code in ultralytics/nn/modules/transformer.py
forward_ffn
Perform forward pass through the Feed-Forward Network part of the layer.
with_pos_embed staticmethod
ultralytics.nn.modules.transformer.DeformableTransformerDecoder
Bases: Module
Implementation of Deformable Transformer Decoder based on PaddleDetection.
Source code in ultralytics/nn/modules/transformer.py
forward
forward(
embed,
refer_bbox,
feats,
shapes,
bbox_head,
score_head,
pos_mlp,
attn_mask=None,
padding_mask=None,
)
Perform the forward pass through the entire decoder.