Experiment Tracking

Experiment tracking is the practice of recording everything about each machine learning training run, including its code version, data version, settings, environment, metrics over time, and output files such as weights and plots, in one searchable store. A project that tries dozens of learning rates, augmentation settings, and model sizes produces runs that look alike on disk, and tracking keeps the configuration behind each result, so the best run can be compared, reproduced, and traced back months later. TensorBoard, released with TensorFlow in 2015, made live training curves standard, and Databricks open-sourced MLflow in 2018 with tracking as one of its three original components. Weights & Biases, Comet, ClearML, and DVC are other widely used trackers.

What a Tracker Records#

Each run gets a unique ID and belongs to an experiment, a named group of related runs. A typical run record holds:

• Hyperparameters: Learning rate, batch size, optimizer, model architecture, random seed, and every other setting that shapes training.
• Metrics: Training and validation losses and scores logged per step or epoch, plus system metrics such as GPU memory and utilization.
• Artifacts: Checkpoints, confusion matrices, sample predictions, and configuration files.
• Lineage: The Git commit, dataset version, package versions, hardware, and author that produced the run.

Logging happens through a client call per value, or automatically through autologging integrations that capture a framework's parameters and metrics with no extra code. A shared tracking server collects runs from every laptop, cloud machine, and CI job in a team, and its dashboard overlays the loss curves of selected runs and filters them by any parameter or metric, which makes problems such as overfitting visible at a glance.

Ultralytics YOLO training writes each run's arguments to args.yaml and its per-epoch losses and metrics to results.csv, and built-in integrations send the same values to MLflow, Comet, ClearML, or DVC as soon as their package is installed, with TensorBoard and Weights & Biases switched on through yolo settings. The project and name training arguments become the experiment and run names in MLflow, so a YOLO26 sweep lands in the tracker already organized.

Comparing and Reproducing Runs#

Dashboards plot a metric against a parameter across runs, or draw parallel-coordinates charts that connect each run's settings to its score, which shows which settings matter in a sweep. In Ultralytics Platform, a project compares its models with overlaid metric and loss curves and a side-by-side table, and the Compare Runs and Pick a Winner lesson turns that comparison into a scripted promotion gate. Reproducing a run takes more than its hyperparameters: the record must also hold the code commit, the exact data snapshot, the package versions, and the random seed, because GPU kernels and data loading order can change results slightly even between otherwise identical runs.

This MLflow script trains a decision tree at four depths on scikit-learn's handwritten digits, records each depth as a parameter and the validation accuracy as a metric, then queries the experiment like a table:

import mlflow
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

# Store runs in a local SQLite file and group them under one experiment
mlflow.set_tracking_uri("sqlite:///mlflow.db")
mlflow.set_experiment("digits-tree-depth")
X_train, X_val, y_train, y_val = train_test_split(*load_digits(return_X_y=True), random_state=0)

# One tracked run per setting: the parameter goes in, the metric comes out
for depth in [2, 4, 8, 16]:
    with mlflow.start_run(run_name=f"depth-{depth}"):
        model = DecisionTreeClassifier(max_depth=depth, random_state=0).fit(X_train, y_train)
        mlflow.log_param("max_depth", depth)
        mlflow.log_metric("val_accuracy", model.score(X_val, y_val))

# Query the experiment like a table, best run first
runs = mlflow.search_runs(order_by=["metrics.val_accuracy DESC"])
print(runs[["params.max_depth", "metrics.val_accuracy"]])

On a fresh database the table lists depth 16 first at 0.838 validation accuracy, then depth 8 at 0.818, depth 4 at 0.533, and depth 2 at 0.318. Running mlflow ui in the same folder with --backend-store-uri sqlite:///mlflow.db opens the same runs in the browser.

Experiment Tracking vs. Related Practices#

Experiment tracking is one stage of machine learning operations (MLOps), next to three practices that are often confused with it:

• Model registry: Keeps the few trained models worth deploying, each version linked back to the tracked run that produced it.
• Model monitoring: Watches a deployed model's inputs, outputs, and accuracy in production, after tracking has done its work during development.
• Hyperparameter tuning: Search tools such as Optuna and Ray Tune launch many runs automatically, and the tracker stores and ranks every one of them.