Get Started

Callbacks

The Ultralytics framework supports callbacks, which serve as entry points at key stages of the train, val, predict, and export modes. Each callback accepts a Trainer, Validator, Predictor, or Exporter object, depending on the operation type.



Watch: How to use Ultralytics Callbacks | Predict, Train, Validate and Export Callbacks | Ultralytics YOLO26 🚀

Examples#

Returning Additional Information with Prediction#

This example returns the original frame along with each result object:

from ultralytics import YOLO

def on_predict_batch_end(predictor):
    """Combine prediction results with corresponding frames."""
    _, image, _ = predictor.batch

    # Ensure that image is a list
    image = image if isinstance(image, list) else [image]

    # Combine the prediction results with the corresponding frames
    predictor.results = zip(predictor.results, image)

# Create a YOLO model instance
model = YOLO("yolo26n.pt")

# Add the custom callback to the model
model.add_callback("on_predict_batch_end", on_predict_batch_end)

# Iterate through the results and frames
for result, frame in model.predict():  # or model.track()
    pass

Registering PyTorch Forward Hooks Before Training#

model.train() builds a training model from the model configuration and transfers the loaded weights into it. PyTorch hooks registered directly on model.model before this call are runtime state, so the rebuilt training model does not inherit them. Register training hooks in on_pretrain_routine_end, after model setup is complete:

from ultralytics import YOLO
from ultralytics.utils.torch_utils import unwrap_model

def preprocess_input(module, inputs):
    """Center each input channel before the first model layer."""
    images = inputs[0]
    return (images - images.mean(dim=(-2, -1), keepdim=True),)

def register_forward_pre_hook(trainer):
    """Attach preprocessing to the active training model."""
    train_model = unwrap_model(trainer.model)
    train_model.model[0].register_forward_pre_hook(preprocess_input)

model = YOLO("yolo26n.pt")
model.add_callback("on_pretrain_routine_end", register_forward_pre_hook)
model.train(data="coco8.yaml", epochs=1)

unwrap_model() handles both single-device and DistributedDataParallel training. Do not attach a locally defined hook to trainer.ema.ema, because training checkpoints serialize the EMA model and another process may not be able to import the callback when loading the checkpoint. If the same preprocessing must run during training validation, implement it as an importable model component instead of a runtime hook. NaN recovery rebuilds the EMA from the training model, so a hook registered this way is copied into the EMA and saved checkpoints after a recovery; define the hook function in an importable module if checkpoints must load in another process.

Standalone model.val() copies the loaded model for each call. Prediction creates and caches a copy on its first call, so register hooks on model.model before the first model.predict() or model.track() call; hooks added afterward do not reach the cached predictor. Register runtime hooks again after loading a checkpoint in a new process.

Access Model metrics using the on_model_save callback#

This example shows how to retrieve training details, such as the best_fitness score, total_loss, and other metrics after a checkpoint is saved using the on_model_save callback.

from ultralytics import YOLO

# Load a YOLO model
model = YOLO("yolo26n.pt")

def print_checkpoint_metrics(trainer):
    """Print trainer metrics and loss details after each checkpoint is saved."""
    print(
        f"Model details\n"
        f"Best fitness: {trainer.best_fitness}, "
        f"Loss names: {trainer.loss_names}, "  # List of loss names
        f"Metrics: {trainer.metrics}, "
        f"Total loss: {trainer.tloss}"  # Total loss value
    )

if __name__ == "__main__":
    # Add on_model_save callback.
    model.add_callback("on_model_save", print_checkpoint_metrics)

    # Run model training on custom dataset.
    results = model.train(data="coco8.yaml", epochs=3)

All Callbacks#

Below are all the supported callbacks. For more details, refer to the callbacks source code.

Trainer Callbacks#

CallbackDescription
on_pretrain_routine_startTriggered at the beginning of the pre-training routine, before data loading and model setup.
on_pretrain_routine_endTriggered at the end of the pre-training routine, after data loading and model setup are complete.
on_train_startTriggered when the training starts, before the first epoch begins.
on_train_epoch_startTriggered at the start of each training epoch, before batch iteration begins.
on_train_batch_startTriggered at the start of each training batch, before the forward pass.
optimizer_stepTriggered during the optimizer step. Reserved for custom integrations; not called by the default training loop.
on_before_zero_gradTriggered before gradients are zeroed. Reserved for custom integrations; not called by the default training loop.
on_train_batch_endTriggered at the end of each training batch, after the backward pass. The optimizer step may be deferred due to gradient accumulation.
on_train_epoch_endTriggered at the end of each training epoch, after all batches are processed but before validation. Validation metrics and fitness may not be available yet.
on_model_saveTriggered when the model checkpoint is saved, after validation.
on_fit_epoch_endTriggered at the end of each fit epoch (train + val), after validation and any checkpoint save. Validation metrics are available, and fitness is available for the per-epoch training call. This callback is also called during final best-model evaluation, where no checkpoint save occurs and fitness may not be present.
on_train_endTriggered when the training process ends, after final evaluation of the best model.
on_params_updateTriggered when model parameters are updated. Reserved for custom integrations; not called by the default training loop.
teardownTriggered when the training process is being cleaned up.

Validator Callbacks#

CallbackDescription
on_val_startTriggered when validation starts.
on_val_batch_startTriggered at the start of each validation batch.
on_val_batch_endTriggered at the end of each validation batch.
on_val_endTriggered when validation ends.

Predictor Callbacks#

CallbackDescription
on_predict_startTriggered when the prediction process starts.
on_predict_batch_startTriggered at the start of each prediction batch.
on_predict_postprocess_endTriggered at the end of prediction post-processing.
on_predict_batch_endTriggered at the end of each prediction batch.
on_predict_endTriggered when the prediction process ends.

Exporter Callbacks#

CallbackDescription
on_export_startTriggered when the export process starts.
on_export_endTriggered when the export process ends.

FAQ#

  • Ultralytics callbacks are entry points that are triggered during key stages of model operations such as training, validation, prediction, and export. They let you add custom functionality at specific points in the process without modifying the library. Each callback accepts a Trainer, Validator, Predictor, or Exporter object, depending on the operation type.

    To use a callback, define a function and add it to the model using the model.add_callback() method. Here is an example of returning additional information during prediction:

    from ultralytics import YOLO
    
    def on_predict_batch_end(predictor):
        """Handle prediction batch end by combining results with corresponding frames; modifies predictor results."""
        _, image, _ = predictor.batch
        image = image if isinstance(image, list) else [image]
        predictor.results = zip(predictor.results, image)
    
    model = YOLO("yolo26n.pt")
    model.add_callback("on_predict_batch_end", on_predict_batch_end)
    for result, frame in model.predict():
        pass
  • Customize your Ultralytics training routine by injecting logic at specific stages of the training process. Ultralytics YOLO provides a variety of training callbacks, such as on_train_start, on_train_end, and on_train_batch_end, which allow you to add custom metrics, processing, or logging.

    Here's how to log the learning rate of each optimizer parameter group at the end of every training epoch:

    from ultralytics import YOLO
    
    def log_learning_rates(trainer):
        """Print the current learning rates after each training epoch."""
        print(f"Epoch {trainer.epoch + 1}: {trainer.lr}")
    
    model = YOLO("yolo26n.pt")
    model.add_callback("on_train_epoch_end", log_learning_rates)
    model.train(data="coco8.yaml", epochs=3)
    Tip

    You don't need a callback to freeze BatchNorm statistics when using freeze: the trainer automatically puts the BatchNorm layers of frozen layers in eval mode at the start of every epoch.

    For more details on effectively using training callbacks, see the Training Guide.

  • Validation callbacks let you add custom processing, logging, or metrics calculation to model evaluation. Callbacks like on_val_start, on_val_batch_end, and on_val_end provide entry points to inject custom logic at each stage of validation.

    For example, to plot all validation batches instead of just the first three:

    import inspect
    
    from ultralytics import YOLO
    
    def plot_samples(validator):
        frame = inspect.currentframe().f_back.f_back
        v = frame.f_locals
        validator.plot_val_samples(v["batch"], v["batch_i"])
        validator.plot_predictions(v["batch"], v["preds"], v["batch_i"])
    
    model = YOLO("yolo26n.pt")
    model.add_callback("on_val_batch_end", plot_samples)
    model.val(data="coco.yaml")

    For more insights on incorporating callbacks into your validation process, see the Validation Guide.

  • To attach a custom callback for prediction mode in Ultralytics YOLO, define a callback function and register it with the prediction process. Common prediction callbacks include on_predict_start, on_predict_batch_end, and on_predict_end. These allow for the modification of prediction outputs and the integration of additional functionalities, like data logging or result transformation.

    Here is an example where a custom callback saves predictions based on whether an object of a particular class is present:

    from ultralytics import YOLO
    
    model = YOLO("yolo26n.pt")
    
    class_id = 2
    
    def save_on_object(predictor):
        r = predictor.results[0]
        if class_id in r.boxes.cls:
            predictor.args.save = True
        else:
            predictor.args.save = False
    
    model.add_callback("on_predict_postprocess_end", save_on_object)
    results_stream = model("pedestrians.mp4", stream=True, save=True)
    
    for result in results_stream:
        pass

    For more comprehensive usage, refer to the Prediction Guide, which includes detailed instructions and additional customization options.

  • Ultralytics YOLO supports various practical implementations of callbacks to enhance and customize different phases like training, validation, and prediction. Some practical examples include:

    • Logging Custom Metrics: Log additional metrics at different stages, such as at the end of training or validation epochs.
    • Data Augmentation: Implement custom data transformations or augmentations during prediction or training batches.
    • Intermediate Results: Save intermediate results, such as predictions or frames, for further analysis or visualization.

    Example: Combining frames with prediction results during prediction using on_predict_batch_end:

    from ultralytics import YOLO
    
    def on_predict_batch_end(predictor):
        """Combine prediction results with frames."""
        _, image, _ = predictor.batch
        image = image if isinstance(image, list) else [image]
        predictor.results = zip(predictor.results, image)
    
    model = YOLO("yolo26n.pt")
    model.add_callback("on_predict_batch_end", on_predict_batch_end)
    for result, frame in model.predict():
        pass

    Explore the callback source code for more options and examples.

Comments15

Ask a question, share your results, or help someone else.

  1. drbilal216GitHub

    Hi can i and how use callback to freeze initial layers for number of epochs and unfreeze after that, is there any builtin function for that? i used freeze builtin function but it freezes for all the epochs.

    1. drbilal216GitHub

      also, after certain epochs the change in model training is minimal can we use callback then to change lr, how and what parameter to change ?

    2. Glenn JocherUltralytics

      @drbilal216 yes, you can use callbacks to freeze/unfreeze layers dynamically during training. Here's how to freeze initial layers for specific epochs then unfreeze them:

      from ultralytics import YOLO
      
      def dynamic_freeze_callback(trainer):
          freeze_epochs = 10  # Freeze for first 10 epochs
          freeze_layers = 10  # Number of layers to freeze
          
          if trainer.epoch < freeze_epochs:
              # Freeze layers
              for i, (name, param) in enumerate(trainer.model.named_parameters()):
                  if i < freeze_layers:
                      param.requires_grad = False
          else:
              # Unfreeze all layers
              for param in trainer.model.parameters():
                  param.requires_grad = True
      
      model = YOLO("yolo11n.pt")
      model.add_callback("on_train_epoch_start", dynamic_freeze_callback)
      model.train(data="coco8.yaml", epochs=20)

      For dynamic learning rate adjustment when training plateaus, you can modify the scheduler within a callback:

      def dynamic_lr_callback(trainer):
          if trainer.epoch > 50 and trainer.epoch % 10 == 0:  # Every 10 epochs after epoch 50
              for param_group in trainer.optimizer.param_groups:
                  param_group['lr'] *= 0.9  # Reduce LR by 10%
      
      model.add_callback("on_train_epoch_end", dynamic_lr_callback)
      1
    3. drbilal216GitHub

      @glenn-jocher Thanks for the code! I'm working on a low-quality PPE dataset. With a custom YAML, I improved YOLOv12n accuracy by 3%, and got similar results on YOLOv11n with better FPS.

      The default Nano model lets me use pretrained=True, but custom architectures start from scratch. Still, I can reuse best.pt as pretrained weights on the same YAML later.

      I'm now exploring argument tuning and callbacks for further improvements. I also tried freezing/unfreezing layers mid-training but saw no gains.

      Lastly, I integrated Flash into YOLOv12, but I don’t see any sign that it’s being used during training (tested on Colab and A100 80GB-server).

  2. jnance34GitHub

    Hello,

    I'm trying to implement a custom on_pretrain_routine_end callback with a model.tune() call, but it appears that the Tuner class isn't adding my custom callback function to the Tuner.callbacks list. Here's some pseudo code of what I'm trying to do:

    def my_custom_callback(trainer):
        print("I'm in my custom callback function!")
    
    model = YOLO("model/yolo11n.pt")
    model.add_callback('on_pretrain_routine_end', my_custom_function)
    model.tune(...)
    # Expect to see "I'm in my custom callback function" for each trial run...

    For some reason my custom callback function isn't getting called for each training run of the Tuner. Am I missing a step here? I've taken a look at the source code but don't see anything obviously wrong...

    Thanks!

    -James

  3. ruslanraupoffGitHub

    When I'm using single device add_callback works well but multi gpu's not working any solution for this?

    1. Glenn JocherUltralytics

      @ruslanraupoff for multi-GPU training with callbacks, ensure your callback function is properly registered in all processes. When using DDP mode, callbacks need to be added before the training starts and should be compatible with distributed training environments. If you're experiencing issues, verify that your callback logic is thread-safe and doesn't rely on process-specific resources. For implementation details, refer to our callbacks documentation: https://docs.ultralytics.com/usage/callbacks

      1
  4. D3RK0GitHub

    Hi, in the documentation above, I see that predictor.batch is used, which I assume refers to the current batch of the predictor. Is there a similar trainer.batch for a trainer? I need it in order to apply data augmentation during the training of my YOLOv8 classifier.

    1. Glenn JocherUltralytics(edited)

      @D3RK0 trainer callbacks only receive the trainer object, so the current batch is not available there as trainer.batch. To apply custom augmentation to each training batch, subclass your trainer (e.g. ClassificationTrainer) and override preprocess_batch(self, batch), which receives every training batch before the forward pass.

    2. D3RK0GitHub(edited)

      @glenn-jocher Many thanks for your quick response!

      I am dealing with following problem though:

      Problem

      When I do trainer.batch, I get the following error:

      Traceback (most recent call last):
        File "C:\Program Files\JetBrains\PyCharm Community Edition 2022.3.3\plugins\python-ce\helpers\pydev\pydevd.py", line 1534, in _exec
          pydev_imports.execfile(file, globals, locals)  # execute the script
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Program Files\JetBrains\PyCharm Community Edition 2022.3.3\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
          exec(compile(contents+"\n", file, 'exec'), glob, loc)
        File "C:\Users\derk_\Documents\Universiteit\IRP\Research_Project\main.py", line 397, in <module>
          main()
        File "C:\Users\derk_\Documents\Universiteit\IRP\Research_Project\main.py", line 377, in main
          model.train(mode="classify",
        File "C:\Users\derk_\anaconda3\envs\cobra\Lib\site-packages\ultralytics\engine\model.py", line 802, in train
          self.trainer.train()
        File "C:\Users\derk_\anaconda3\envs\cobra\Lib\site-packages\ultralytics\engine\trainer.py", line 207, in train
          self._do_train(world_size)
        File "C:\Users\derk_\anaconda3\envs\cobra\Lib\site-packages\ultralytics\engine\trainer.py", line 368, in _do_train
          self.run_callbacks("on_train_batch_start")
        File "C:\Users\derk_\anaconda3\envs\cobra\Lib\site-packages\ultralytics\engine\trainer.py", line 168, in run_callbacks
          callback(self)
        File "C:\Users\derk_\Documents\Universiteit\IRP\Research_Project\main.py", line 185, in batch_augmentation_callback
          batch = trainer.batch  # Access the current batch
                  ^^^^^^^^^^^^^
      AttributeError: 'ClassificationTrainer' object has no attribute 'batch'

      My code

      model = YOLO("yolov8m-cls.pt")
      
      model.add_callback("on_train_batch_start", batch_augmentation_callback)
      
      model.train(mode="classify",
                  data=data_file_dir,
                  epochs=args.n_epochs)
      
      # Where batch_augmentation_callback is my callback function
      def batch_augmentation_callback(trainer):
          batch = trainer.batch                                        # This code line causes the error reported above
          images, labels = batch['img'], batch['label']
          # More code below to create data augmentation

      Content of trainer object that is passed into batch_augmentation_callback (my callback function)

      • trainer = {ClassificationTrainer} <ultralytics.models.yolo.classify.train.ClassificationTrainer object at 0x0000016929FA5010>
      • accumulate = {int} 4
      • amp = {bool} False
      • args = {IterableSimpleNamespace} IterableSimpleNamespace(task='classify', mode='train', model='yolov8m-cls.pt', data='C:/Users/derk_/Documents/Universiteit/IRP...ugment='randaugment', erasing=0.4, crop_fraction=1.0, cfg=None, tracker='botsort.yaml', save_dir='C:\runs\classify\train53')
      • batch_size = {int} 16
      • best = {WindowsPath} WindowsPath('C:/runs/classify/train53/weights/best.pt')
      • best_fitness = {NoneType} None
      • callbacks = {defaultdict: 25} defaultdict(<class 'list'>, {'on_pretrain_routine_start': [<function on_pretrain_routine_start at 0x0000016929C79440>, <functi...C7A2A0>, <function on_export_start at 0x000001692A55C220>], 'on_export_end': [<function on_export_end at 0x0000016929C7A340>]})
      • csv = {WindowsPath} WindowsPath('C:/runs/classify/train53/results.csv')
      • data = {dict: 5} {'names': {0: '1', 1: '10', 2: '100', 3: '101', 4: '102', 5: '11', 6: '12', 7: '13', 8: '14', 9: '15', 10: '16', 11: '17', 12: '18', 13: '19', 14: '2', 15: '20', 16: '21', 17: '22', 18: '23', 19: '24', 20: '25', 21: '26', 22: '27', 23: '28', 24: '29', 25:
      • device = {device} device(type='cpu')
      • ema = {ModelEMA} <ultralytics.utils.torch_utils.ModelEMA object at 0x000001692AC80C90>
      • epoch = {int} 0
      • epoch_time = {NoneType} None
      • epoch_time_start = {float} 1731355033.8569636
      • epochs = {int} 1
      • fitness = {NoneType} None
      • hub_session = {NoneType} None
      • last = {WindowsPath} WindowsPath('C:/runs/classify/train53/weights/last.pt')
      • loss = {NoneType} None
      • loss_names = {list: 1} ['loss']
      • metrics = {dict: 3} {'metrics/accuracy_top1': 0, 'metrics/accuracy_top5': 0, 'val/loss': 0}
      • model = {ClassificationModel} ClassificationModel(\n (model): Sequential(\n (0): Conv(\n (conv): Conv2d(3, 48, kernel_size=(3, 3), stride=(2, 2), padd...1)\n (drop): Dropout(p=0.0, inplace=True)\n (linear): Linear(in_features=1280, out_features=102, bias=True)\n )\n )\n)
      • optimizer = {AdamW} AdamW (\nParameter Group 0\n amsgrad: False\n base_momentum: 0.85\n betas: (0.9499811807836418, 0.999)\n capturable: Fa... lr: 0.004018066447703833\n max_lr: 0.1\n max_momentum: 0.95\n maximize: False\n min_lr: 4e-07\n weight_decay: 0.0\n)
      • plot_idx = {list: 6} [0, 1, 2, -3465, -3464, -3463]
      • plots = {dict: 0} {}
      • resume = {bool} False
      • save_dir = {WindowsPath} WindowsPath('C:/runs/classify/train53')
      • save_period = {int} 1
      • scaler = {GradScaler} <torch.amp.grad_scaler.GradScaler object at 0x000001692921A0D0>
      • scheduler = {OneCycleLR} <torch.optim.lr_scheduler.OneCycleLR object at 0x000001692ACF5210>
      • start_epoch = {int} 0
      • stop = {bool} False
      • stopper = {EarlyStopping} <ultralytics.utils.torch_utils.EarlyStopping object at 0x000001692ACF7190>
      • stride = {int} 32
      • test_loader = {InfiniteDataLoader: 32} <ultralytics.data.build.InfiniteDataLoader object at 0x000001692AAD7B50>
      • testset = {WindowsPath} WindowsPath('C:/Users/derk_/Documents/Universiteit/IRP/Research_Project/Oxford-Flower-dataset/jpg/val')
      • tloss = {NoneType} None
      • train_loader = {InfiniteDataLoader: 385} <ultralytics.data.build.InfiniteDataLoader object at 0x00000169297FFDD0>
      • train_time_start = {float} 1731355033.8569636
      • trainset = {WindowsPath} WindowsPath('C:/Users/derk_/Documents/Universiteit/IRP/Research_Project/Oxford-Flower-dataset/jpg/train')
      • validator = {ClassificationValidator} <ultralytics.models.yolo.classify.val.ClassificationValidator object at 0x000001692AC80BD0>
      • wdir = {WindowsPath} WindowsPath('C:/runs/classify/train53/weights')

      Question

      If it is right that trainer.batch does not exist (in this case), which other object in trainer - as listed above in the content of trainer - can I extract the current batch?

    3. Glenn JocherUltralytics(edited)

      The trainer.batch attribute does not exist in the current implementation, and iterating trainer.train_loader inside a callback would draw a different batch rather than the current one. Instead, override preprocess_batch(self, batch) in a custom trainer subclass to modify each batch before it is used.

    4. D3RK0GitHub(edited)

      @glenn-jocher Many thanks for answering!

      Okay, I have been able to access the batches via trainer.train_loader and subsequently apply data augmentation to each batch. However, ...

      Problem

      ... assigning the batch back (see also the code below) to trainer.train_loader is not possible, as I get:

      trainer.train_loader[i] = batch ~~~~~~~~~~~~~~~~~~~~^^^ TypeError: 'InfiniteDataLoader' object does not support item assignment

      Code

      def batches_augmentation_callback(trainer):
          for i, batch in enumerate(trainer.train_loader):
              images, labels = batch['img'], batch['cls']
              cutmix_or_mixup = LoadOxfordFlowerDataset.data_augmentation(n_classes_in_batch=args.n_classes)
              images, labels = cutmix_or_mixup(images, labels)
              batch = {"img": images, "cls": labels}
              trainer.train_loader[i] = batch     # BUG: trainer.train_loader object does not support item assignment

      Question

      How can I return the augmented batch to the trainer object if assignment does not work?

    5. Glenn JocherUltralytics

      You cannot directly assign to trainer.train_loader. Instead, apply your augmentation within the loop and yield the modified batch back to the training process. Consider using a custom data loader or modifying the dataset directly for more complex augmentations.