Automated Machine Learning (AutoML)

Automated machine learning (AutoML) is the automation of the trial-and-error work in building a machine learning (ML) model: preparing features, choosing an algorithm, setting its hyperparameters, and sometimes designing a network architecture or combining several models. Given a dataset, a target metric, and a compute budget, an AutoML system searches over candidate pipelines, scores each on held-out data, and returns the best one it finds. The Auto-WEKA project framed the core of this task as the combined algorithm selection and hyperparameter optimization (CASH) problem, a formulation that later tools such as auto-sklearn adopted. AutoML gives non-specialists strong baselines and saves experts the repetitive part of experimentation.

How AutoML Works#

An AutoML system has three ingredients. The search space defines what can vary, from data preprocessing and feature engineering steps to the model family, such as linear regression, XGBoost, or a neural network, and each family's settings, such as the learning rate. The search strategy picks which candidates to try, using random search, Bayesian optimization that models which regions of the space look promising, or evolutionary algorithms that mutate and recombine good configurations. The evaluation step scores each candidate on validation data or with cross-validation. Many systems also warm-start the search with meta-learning from earlier datasets and finish by combining the best candidates into a model ensemble.

Training every candidate to completion is the main cost, so multi-fidelity methods first train many configurations cheaply and spend more only on the promising ones. Successive halving, the building block of Hyperband, repeats one round until a single configuration remains:

A large search can also overfit the validation data, since the winner is partly the luckiest of many candidates, which is why the final model is scored on a separate test set.

AutoML Tools#

• AutoGluon: An open-source library started at Amazon that trains many model types on tabular, text, image, and time series data and stacks them into an ensemble.
• auto-sklearn: Searches scikit-learn pipelines with Bayesian optimization, meta-learning, and ensemble construction.
• FLAML: A Microsoft library that targets low-cost search for tabular models and hyperparameters.
• Optuna: A hyperparameter optimization framework whose pruners stop unpromising trials early, often used as the search engine inside custom AutoML loops.
• Google Cloud AutoML: A managed service, now part of Gemini Enterprise Agent Platform, formerly Vertex AI, that trains image classification, object detection, and tabular models without code.
• Amazon SageMaker Autopilot: AWS's AutoML service, whose interface now lives in SageMaker Canvas.

For small tables, tabular foundation models offer a different route to the same goal, predicting from the training rows in context without a per-dataset search.

Automated Tuning for YOLO#

For computer vision models with a fixed, pretrained architecture, automated search usually targets the training recipe. Ultralytics YOLO's model.tune() evolves 26 training hyperparameters by default, including learning rate, momentum, loss gains, and augmentation strengths: each iteration mutates and recombines the best configurations found so far, trains briefly, and records the fitness, and the run saves best_hyperparameters.yaml with plots of fitness across iterations. Passing use_ray=True hands the search to Ray Tune instead.

from ultralytics import YOLO

# Load a pretrained YOLO26 model to tune
model = YOLO("yolo26n.pt")

# Evolve hyperparameters: each iteration mutates the best configurations so far, trains, and scores fitness
model.tune(data="coco8.yaml", epochs=30, iterations=10, plots=False, save=False, val=False)

AutoML vs. Related Terms#

• Hyperparameter tuning: Optimizes the settings of one chosen model. AutoML usually includes it as one stage of a wider search that also chooses the model.
• Neural architecture search (NAS): The branch of AutoML that designs network architectures, such as layer types, depths, and connections.
• Transfer learning: Starts from a pretrained model rather than from scratch. AutoML systems often search over pretrained backbones, which makes the two complementary.
• Machine learning operations (MLOps): Covers the whole lifecycle around a model, including deployment, monitoring, and retraining, and often runs AutoML as one step of a training pipeline.