Behavioral Cloning

Behavioral cloning (BC) is the simplest form of imitation learning: an agent learns to act by training a policy, with ordinary supervised learning, to predict the action an expert took in each situation of a recorded demonstration. The training data is a set of observation-action pairs, such as camera frames paired with a driver's steering angle or a robot's joint readings paired with a teleoperator's commands, and the policy is fit with a regression loss for continuous actions or a classification loss for discrete ones. Behavioral cloning needs no reward function and no interaction with the environment during training, which makes it the usual starting point for teaching robots, vehicles, and software AI agents from human demonstrations.

Origins and Classic Examples#

The name comes from work by Donald Michie and colleagues around 1990 on reconstructing human control skills. Its best-known early demonstration, "Learning to Fly" by Claude Sammut and colleagues in 1992, logged the control actions of people flying a flight simulator and induced decision-tree autopilots from the logs. Dean Pomerleau's ALVINN at Carnegie Mellon had already shown in 1989 that a small neural network could steer a test vehicle from camera images.

NVIDIA's 2016 end-to-end driving system trained a convolutional neural network on about 72 hours of human driving to map raw pixels from a front-facing camera directly to steering commands, and on a typical drive in Monmouth County, New Jersey, it steered on its own about 98% of the time. With discrete actions, behavioral cloning is image classification: an Ultralytics YOLO26 classification model trained on camera frames sorted into one folder per action the driver took, such as left, straight, and right, outputs the cloned action as its top-1 class.

The Compounding Error Problem#

During training, the policy sees only the states the expert visited. At deployment it acts in closed loop, so each small mistake moves it into a state slightly unlike anything in the demonstrations, where it is more likely to err again, and the errors accumulate along the trajectory. This mismatch between training and deployment states is a form of covariate shift. Stéphane Ross and Drew Bagnell showed in 2010 that a cloned policy's expected cost can grow with the square of the task horizon even when its per-step error is small. DAgger (Dataset Aggregation), by Ross, Geoffrey Gordon, and Bagnell in 2011, reduced the growth to linear: the learned policy acts, the expert labels the states it visits with the correct actions, and the policy is retrained on every label collected so far.

This example clones a hand-tuned controller for the Gymnasium CartPole task, which must keep a pole balanced on a moving cart for up to 500 steps, and then applies two rounds of DAgger:

import gymnasium as gym
import numpy as np
from sklearn.tree import DecisionTreeClassifier

env = gym.make("CartPole-v1")
expert = lambda obs: int(obs[2] + 0.5 * obs[3] > 0)  # push toward the side the pole leans and falls


def rollout(policy, seed):
    """Run one episode with a policy and return the states it visited."""
    obs, _ = env.reset(seed=seed)
    states, done = [], False
    while not done:
        states.append(obs)
        obs, _, terminated, truncated, _ = env.step(policy(obs))
        done = terminated or truncated
    return states


states = rollout(expert, seed=0)  # one expert demonstration
for i in range(3):
    # Behavioral cloning: fit a classifier from states to the expert's actions
    clone = DecisionTreeClassifier(max_depth=5, random_state=0).fit(states, [expert(s) for s in states])
    policy = lambda obs: int(clone.predict(obs[None])[0])
    steps = np.mean([len(rollout(policy, seed)) for seed in range(100, 110)])
    print(f"Round {i}: {len(states)} labeled states, clone balances for {steps:.0f} steps")
    states += rollout(policy, seed=i + 1)  # DAgger: the expert labels the states the clone visits

Round 0 is plain behavioral cloning. Trained on one 500-step demonstration from an expert that balances the pole for all 500 steps, the clone keeps the pole up for an average of 30 steps in new episodes. Round 1 adds the 23 states that the round-0 clone visited in one more episode, labeled by the expert, and raises the average to 135 steps; after 70 more such states in round 2, the clone balances for the full 500.

Other fixes reshape the demonstrations themselves. NVIDIA's driving system added images from left and right cameras and synthetic shifts and rotations of the camera view, each labeled with the steering that would return the car to the lane center within two seconds, a form of data augmentation that teaches recovery. DART injects noise into the demonstrator's actions while recording, so the data includes the expert correcting small deviations. Action chunking predicts several future actions at once, which shortens the effective horizon over which errors compound; the ACT policy learned fine-grained bimanual tasks, such as slotting a battery, with 80% to 90% success from 10 minutes of demonstrations.

Multimodal Demonstrations and Modern Policies#

When demonstrators solve the same situation in different ways, such as passing an obstacle on the left or on the right, a regression policy predicts their average, which may be an action nobody took. Google's RT-1, a transformer trained by behavioral cloning on 130,000 robot demonstrations, can represent several modes because it discretizes each action dimension into 256 bins and predicts them as tokens. Diffusion policies model the full distribution of actions by denoising random noise, and they have become a common choice for embodied AI manipulation.

Language models are trained the same way. Supervised fine-tuning of a large language model (LLM) on human-written answers, the core of instruction tuning, is behavioral cloning over text, and OpenAI's WebGPT paper used exactly that name for the stage before reinforcement learning from human feedback (RLHF).

Behavioral Cloning vs. Related Terms#

• Reinforcement learning: Learns from a reward signal through trial and error and can surpass the demonstrator, which behavioral cloning cannot. Offline RL methods such as TD3+BC add a behavioral cloning term to keep the learned policy close to the recorded data.
• Inverse reinforcement learning: Infers the reward function that explains the demonstrations and then optimizes it. Generative adversarial imitation learning (GAIL) trains a policy until a discriminator cannot tell its behavior from the expert's.
• Knowledge distillation: Trains a small student model to match a larger teacher model's outputs. Behavioral cloning of a trained policy instead of a human is called policy distillation.