Online Learning

Online learning is a machine learning setting in which a model learns from data that arrives in sequence, updating after each example or small batch instead of training once on a fixed dataset. At every step the model makes a prediction, receives the true outcome, measures its loss, and adjusts its parameters before the next example arrives. The approach suits data that comes as a stream, such as clicks, transactions, or sensor readings, data too large to store, and relationships that change over time. Batch or offline learning is the opposite: the model trains on all of its training data at once and stays fixed until it is retrained.

How Online Learning Works#

Frank Rosenblatt's perceptron of 1958 was an online learner, nudging its weights after every misclassified example, and stochastic gradient descent (SGD) applies the same per-example or per-batch update to almost any differentiable model. Other algorithms are built for streams: Hoeffding trees, introduced by Pedro Domingos and Geoff Hulten in 2000, grow a decision tree from a stream while seeing each example only once, and libraries such as River implement streaming versions of common models. In scikit-learn, estimators with a partial_fit method learn incrementally.

Online learners are usually evaluated prequentially, a term from statistician Philip Dawid: each new example is first used to test the model and then to train it, so the running score is computed on data the model had not yet seen. Theory measures an online learner by its regret, the gap between its cumulative loss and that of the best fixed model chosen in hindsight. Martin Zinkevich showed in 2003 that online gradient descent keeps regret growing only with the square root of the number of steps, so the average gap per step shrinks toward zero.

This example streams 400 mini-batches into a logistic regression trained with SGD. Halfway through, the rule that assigns labels changes, and a copy frozen at that moment stands in for a model that stopped learning:

import copy

import numpy as np
from sklearn.linear_model import SGDClassifier

rng = np.random.default_rng(0)
online = SGDClassifier(loss="log_loss", random_state=0)
scores = {"frozen": [], "online": []}

for step in range(400):  # 400 mini-batches of 20 examples arrive one at a time
    X = rng.normal(size=(20, 2))
    w = [1.0, 1.0] if step < 200 else [1.0, -1.0]  # the labeling rule changes at step 200
    y = (X @ np.array(w) > 0).astype(int)
    if step == 200:
        frozen = copy.deepcopy(online)  # a copy that stops learning when the rule changes
    if step >= 200:  # test on each new batch before learning from it
        scores["frozen"].append(frozen.score(X, y))
        scores["online"].append(online.score(X, y))
    online.partial_fit(X, y, classes=[0, 1])

for name, acc in scores.items():
    print(f"{name}: {np.mean(acc):.2f} accuracy after the change")

After the change, the frozen model scores 0.47, no better than guessing, while the online model averages 0.98. The online model scores 0.35 on the first batch under the new rule and 0.80 on the second, then recovers within a few more batches.

Concept Drift and Forgetting#

The main reason to learn online is concept drift, a change in the relationship between inputs and labels, often accompanied by data drift in the inputs themselves. Drift can be sudden, as in the example, or it can build gradually or recur with seasons and time of day. Detectors such as ADWIN, from Albert Bifet and Ricard Gavaldà, watch the error stream and signal when its average changes, which can trigger faster updates or a reset.

Adapting fast has a cost. A high learning rate follows drift quickly but also follows noise, and a network that only sees recent data overwrites what it learned earlier, a failure called catastrophic forgetting. Replay buffers that mix stored older examples into each update are a common remedy.

Online Learning in Practice#

Online learning is common where labels arrive within seconds. Ad click prediction is the classic case: Google described in 2013 how its click-through models train continuously on streaming data with the FTRL-Proximal algorithm, and recommendation systems and fraud scoring often update the same way. Deployed vision models are usually updated in batches instead, because labels arrive late and one corrupted update can degrade every later prediction, so teams combine model monitoring with periodic fine-tuning on recent and replayed data, as in model maintenance workflows.

Smaller online updates do run inside vision systems. In Ultralytics YOLO26 track mode, enabling re-identification with with_reid: True makes the default TrackTrack tracker keep an appearance embedding for every track and update it after each matched detection as a moving average, giving the new embedding a weight of 5% times the detection's confidence, so each track's appearance follows gradual changes in pose and lighting while the ReID network stays fixed.

Continual learning overlaps with online learning but focuses on retaining earlier tasks while new ones arrive, whereas online learning describes how often updates happen. Active learning chooses which examples to label, and reinforcement learning also learns from a stream of interactions, but from rewards for its own actions rather than from labeled examples.