Sparse Autoencoders (SAE)

A sparse autoencoder (SAE) is an autoencoder trained to reconstruct its input through a hidden layer in which only a few units are active for any given input. In current AI research, SAEs are trained on the internal activations of a large language model (LLM) or vision model, with many more hidden units than input dimensions, so that each hidden unit, called a feature or latent, tends to fire for one human-recognizable concept. They are a central tool of mechanistic interpretability: Anthropic's 2024 Scaling Monosemanticity work trained SAEs with up to 34 million features on Claude 3 Sonnet's activations and found features for concepts such as the Golden Gate Bridge, code errors, and deception.

Why Sparse Features Help Interpretability#

Individual neurons in large models are often polysemantic, responding to several unrelated concepts. The superposition hypothesis, studied in Anthropic's Toy Models of Superposition, explains why: a layer with d dimensions can represent far more than d concepts as nearly orthogonal directions, as long as only a few of them are active at once. An SAE searches for that overcomplete set of directions. Its sparse code for an activation lists which few directions are present and how strongly, which is far easier to read than thousands of mixed neuron values.

How a Sparse Autoencoder Works#

The encoder maps an activation vector to a much wider vector of latents, followed by a ReLU, and the decoder reconstructs the activation as a weighted sum of its columns, so each decoder column is the direction of one feature. Training minimizes reconstruction error plus a sparsity constraint. The classic choice is an L1 penalty on the latents, a form of regularization; OpenAI's TopK SAEs keep only the k largest latents per input and scaled to a 16-million-latent SAE on GPT-4 activations; and Google DeepMind's JumpReLU SAEs zero every latent below a learned threshold. Once trained, an SAE's latents are put to work in two main ways:

The toy example below packs 64 known feature directions into 32 dimensions, so that no single dimension corresponds to one feature, and trains a TopK SAE to recover them:

import torch
import torch.nn.functional as F

torch.manual_seed(0)
# Toy activations: 64 feature directions packed into 32 dimensions (superposition), ~3 active per sample
features = F.normalize(torch.randn(64, 32), dim=1)
acts = (torch.rand(8192, 64) * (torch.rand(8192, 64) < 0.05)) @ features

encoder, decoder = torch.nn.Linear(32, 256), torch.nn.Linear(256, 32)  # 256 latents: 8x overcomplete
optimizer = torch.optim.Adam([*encoder.parameters(), *decoder.parameters()], lr=1e-3)
for step in range(3000):
    # TopK sparsity: keep each sample's 4 largest latents and zero the rest
    values, idx = encoder(acts).topk(4, dim=-1)
    latents = torch.zeros(len(acts), 256).scatter(-1, idx, F.relu(values))
    loss = (decoder(latents) - acts).pow(2).sum(-1).mean()
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

directions = F.normalize(decoder.weight.T, dim=1)  # one learned direction per latent
print(f"Unexplained variance: {loss.item() / (acts - acts.mean(0)).pow(2).sum(-1).mean().item():.3f}")
print(f"Features matching a single neuron: {(features.abs().max(dim=1).values > 0.9).sum().item()} of 64")
print(f"Features matching an SAE latent: {((directions @ features.T).max(dim=0).values > 0.9).sum().item()} of 64")

With at most 4 active latents per sample, the SAE leaves 7.1% of the variance unexplained. None of the 64 planted features lines up with a single input dimension at a cosine similarity above 0.9, while all 64 line up with the decoder direction of some SAE latent.

Applications#

• Feature discovery: Researchers label each latent by collecting the inputs that activate it most, often with an LLM writing the label, and use the features to audit what a model represents.
• Steering: Clamping a feature to a high value changes behavior, much like adding a steering vector. In May 2024 Anthropic briefly released Golden Gate Claude, a version with the bridge feature amplified so that the bridge came up in almost every answer.
• Circuit tracing: Anthropic's 2025 attribution graphs use cross-layer transcoders, an SAE variant that reads one layer's activations and reconstructs the outputs of later MLP layers, to trace multi-step computations inside Claude 3.5 Haiku.
• Open SAE suites: Google DeepMind's Gemma Scope released JumpReLU SAEs for every layer of Gemma 2 2B and 9B, and Gemma Scope 2 adds SAEs and transcoders for the Gemma 3 models from 270M to 27B parameters. The SAELens library trains and loads such SAEs.
• Vision models: SAEs trained on CLIP and Vision Transformer activations find interpretable visual concepts as well. An SAE needs a dataset of activation vectors, and model.embed() in Ultralytics YOLO26 returns one average-pooled activation vector per image from any layer index, the second-to-last layer by default.

Limitations#

SAEs reconstruct activations imperfectly, and replacing a layer's activations with the reconstruction measurably raises the model's loss. The features they find depend on dictionary size and training run, since wider SAEs split one feature into several finer ones, and some latents never activate at all. In March 2025 the Google DeepMind interpretability team reported that probes built on SAE features underperformed plain linear probes at detecting harmful intent in out-of-distribution prompts, and said it was deprioritizing fundamental SAE research while keeping SAEs as one tool among several.

SAE vs. Related Concepts#

• Standard autoencoder: Squeezes data through a bottleneck narrower than the input for compression or dimensionality reduction. An SAE widens the hidden layer and enforces sparsity to separate concepts.
• Linear probing: Trains a supervised classifier to detect one concept chosen in advance. An SAE is unsupervised and discovers features without labels.
• Representation engineering (RepE): Finds concept directions by contrasting activations on paired prompts, while SAE features come from reconstruction alone.
• Sparse coding: The older dictionary-learning method that SAEs approximate with a single encoder pass instead of an optimization per input. Olshausen and Field (1996) showed that a sparse code learned from natural images produces localized, oriented, edge-like filters resembling the receptive fields of simple cells in the primary visual cortex.