Causal Representation Learning

Causal representation learning (CRL) is the problem of learning, from high-dimensional observations such as pixels, a small set of latent variables that correspond to the real causes that generated the data, together with the causal relationships among them. A model with such a representation can predict what happens when one factor is changed, such as moving a light or pushing an object, and keeps working when correlations in its training data change, a form of data drift. Bernhard Schölkopf, Francesco Locatello, Yoshua Bengio, and colleagues set out the research program in Toward Causal Representation Learning in 2021, combining the representation learning of deep learning with the structural causal models of causal inference, which assume the causal variables are already given.

Why Ordinary Representations Fall Short#

A network trained for image classification maps each image into a latent space whose features predict the label as well as possible on the training distribution, and nothing stops those features from mixing the object with its surroundings. In 2018, Sara Beery, Grant Van Horn, and Pietro Perona showed that classifiers that recognize cows on grassy pastures can fail on a cow standing on a beach, because background had become part of what "cow" meant. This failure, called shortcut learning, follows from dataset bias, and more data from the same distribution does not remove it.

CRL rests on the idea that the world is made of independent causal mechanisms: a change in the real world, such as a new camera, new lighting, or a robot's action, alters only a few mechanisms and leaves the rest unchanged. In a representation whose variables match those mechanisms, such a change touches only a few variables. The synthetic Pendulum dataset used to evaluate CausalVAE, a causal variational autoencoder from 2021, shows the target structure: the pendulum's angle and the light's angle jointly cause the shadow's position and length, and all four are rendered into one image.

Identifiability: When Causal Variables Can Be Recovered#

Many different sets of latent variables can explain the same images, so CRL first asks when the true ones are identifiable. In 2019, Locatello and colleagues proved that learning disentangled factors with unsupervised learning is impossible without assumptions about both the model and the data, and a study of more than 12,000 trained models on seven datasets confirmed that unsupervised methods could not reliably find the true factors. Independent component analysis (ICA), the linear base case, shows what such an assumption buys. If independent factors are mixed linearly, ICA can recover them as long as at most one is Gaussian, because any rotation of independent Gaussian variables is again a set of independent Gaussians:

import numpy as np
from sklearn.decomposition import FastICA

rng = np.random.default_rng(0)
mixing = np.array([[2.0, 1.0], [1.0, 1.0]])  # unknown map from two latent factors to two sensor readings

for name, latents in [
    ("Non-Gaussian factors", np.column_stack([rng.uniform(-1, 1, 5000), rng.laplace(size=5000)])),
    ("Gaussian factors", rng.normal(size=(5000, 2))),
]:
    observations = latents @ mixing.T
    recovered = FastICA(n_components=2, random_state=0).fit_transform(observations)
    # Correlation of each true factor with its best-matching recovered component (1.0 = recovered exactly)
    match = np.abs(np.corrcoef(latents.T, recovered.T)[:2, 2:]).max(axis=1)
    print(f"{name}: {match.round(2)}")

FastICA recovers both non-Gaussian factors with a correlation of 1.0. For Gaussian factors, it returns an arbitrary rotation whose components match each true factor with a correlation of only 0.85.

Causal factors add a second difficulty, since causes and their effects are statistically dependent, so CRL methods replace the independence assumption with extra information about how the data was generated. Data recorded under interventions or in several environments, such as a robot moving one object at a time or medical images from several hospitals, reveals which variables each change affects. Video helps when variables evolve smoothly and actions intervene on some of them, as in the CITRIS method of Phillip Lippe and colleagues in 2022. Paired views help as well: Julius von Kügelgen and colleagues proved at NeurIPS 2021 that contrastive learning on augmented image pairs recovers the content that the augmentations leave unchanged, up to an invertible transformation and separated from the style they alter, even when content and style are causally dependent. Supervised training uses the same content-style split. By default, training an Ultralytics YOLO26 classification model applies random crops that keep 50% to 100% of the image area, horizontal flips, RandAugment, and random erasing with probability 0.4, a data augmentation recipe that changes style while the class label, the content, stays fixed.

CRL vs. Related Terms#

• Causal discovery: Learns the causal graph among variables that are already measured, such as temperature and pressure columns in a table. CRL must learn the variables as well as the graph, starting from pixels.
• Disentangled representation learning: Seeks latent dimensions that each capture one independent factor. CRL allows the factors to cause one another and models those links.
• Self-supervised learning: Supplies the training signals, such as augmented views and temporal order, that many CRL identifiability results rely on.
• Explainable AI (XAI): Explains which inputs drove a trained model's prediction, without requiring the model's internal variables to correspond to real-world causes.