Concept Bottleneck Models

A concept bottleneck model (CBM) is a neural network that first predicts a set of human-named concepts from its input and then predicts the final label using only those concepts. For a knee x-ray, the concepts might be joint space narrowing and bone spurs; for a bird photo, wing color and beak shape. Because every prediction passes through this narrow, labeled layer, a person can see which concepts drove the decision and correct a wrong concept to change the outcome, which makes CBMs a form of explainable AI (XAI) built into the architecture. Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang of Stanford and Google introduced the term in Concept Bottleneck Models at ICML 2020, showing accuracy competitive with standard end-to-end models on two tasks: grading knee osteoarthritis from x-rays and identifying bird species.

How Concept Bottleneck Models Work#

A CBM has two parts: a concept predictor that maps the input, such as an image, to concept values, and a label predictor that maps those concept values to the output. Koh et al. compared three ways to train them:

• Independent: The concept predictor learns from concept annotations and the label predictor from the true concepts, each separately.
• Sequential: The concept predictor is trained first, and the label predictor then learns from its predicted concepts.
• Joint: Both parts train together on a weighted sum of the concept loss and the label loss.

On the Osteoarthritis Initiative data, 36,369 knee x-rays labeled with the Kellgren-Lawrence severity grade, the concepts were 10 ordinal findings that radiologists already assess, such as joint space narrowing, bone spurs, and bone sclerosis. On the Caltech-UCSD Birds dataset of 11,788 photos of 200 species, they were 112 binary attributes such as wing color and beak shape. Every training example needs both concept and label annotations, which is the main cost of the approach.

The bottleneck enables test-time intervention: when an expert sees that a predicted concept is wrong, they replace it with the correct value, and the label predictor recomputes the output. On the x-ray task, supplying the true values of just two concepts cut the error of the predicted severity grade from above 0.4 to about 0.3 RMSE.

This PyTorch example trains an independent CBM on toy data where six binary concepts generate noisy inputs and the label depends only on the concepts, then measures label accuracy as an expert supplies the true values of the least certain concepts:

import torch
from torch import nn

torch.manual_seed(0)
# Toy data: 6 binary concepts generate noisy 20-D inputs, and the label depends only on the concepts
concepts = torch.randint(0, 2, (3000, 6)).float()
inputs = concepts @ torch.randn(6, 20) + 2 * torch.randn(3000, 20)
labels = (concepts @ torch.tensor([3.0, 2, 2, 1, 1, 1]) >= 5).float()
train, test = slice(0, 2000), slice(2000, None)

# Independent training: inputs -> concepts, and concepts -> label, each against its own ground truth
concept_model, label_model = nn.Linear(20, 6), nn.Linear(6, 1)
optimizer = torch.optim.Adam([*concept_model.parameters(), *label_model.parameters()], lr=0.05)
bce = nn.functional.binary_cross_entropy_with_logits
for _ in range(500):
    optimizer.zero_grad()
    loss = bce(concept_model(inputs[train]), concepts[train]) + bce(label_model(concepts[train])[:, 0], labels[train])
    loss.backward()
    optimizer.step()

with torch.no_grad():
    probs = concept_model(inputs[test]).sigmoid()
    print(f"Concept accuracy: {((probs > 0.5) == concepts[test]).float().mean():.3f}")
    for k in range(4):
        # Test-time intervention: replace the k least certain predicted concepts with their true values
        uncertain = (probs - 0.5).abs().argsort(dim=1)[:, :k]
        fixed = probs.scatter(1, uncertain, concepts[test].gather(1, uncertain))
        accuracy = ((label_model(fixed)[:, 0] > 0) == labels[test].bool()).float().mean()
        print(f"Intervene on {k} of 6 concepts: label accuracy {accuracy:.3f}")

The concept predictor gets 0.834 of the 6,000 test concepts right, and the CBM labels 0.819 of the 1,000 test examples correctly. Supplying the true values of the one, two, and three least certain concepts per example raises label accuracy to 0.861, 0.915, and 0.951, with no change to either model.

Where CBMs Are Used#

CBMs suit domains where experts already reason in named findings. In medical image analysis, the x-ray study above used the same findings radiologists record, so a clinician can check each one. Safety monitoring has the same structure: an Ultralytics YOLO26 model trained on the Construction-PPE dataset detects named classes such as helmet, no_helmet, vest, and no_boots, and a compliance rule that reads only those detections becomes a concept bottleneck whose decision a reviewer can trace to a specific missing item.

Later work relaxed the annotation requirement. Label-free CBMs generate a concept list with a language model and score concepts with CLIP, scaling the idea to ImageNet without concept labels, and post-hoc CBMs convert an already trained model by projecting its embeddings onto concept directions.

Limitations#

Concept sets rarely capture everything that determines the label, which caps a CBM's accuracy. When the label predictor receives concept probabilities instead of hard values, those probabilities can carry extra information about the label beyond what each concept means, an effect called concept leakage that makes explanations and interventions less trustworthy. Concept Embedding Models represent each concept as a learned vector to keep accuracy high while still supporting intervention. Poorly defined or correlated concepts produce unreliable explanations, and biased concept annotations pass bias into every decision that depends on them.

CBMs vs. Related Concepts#

• Saliency maps: Highlight the pixels that influenced an already trained black-box model. CBMs explain decisions in named concepts and allow those concepts to be edited.
• Mechanistic interpretability: Reverse-engineers the features and circuits a trained model learned on its own. CBMs fix the intermediate concepts in advance through supervision.
• Linear probing: Tests whether a concept can be decoded from a trained model's activations without changing how it predicts. Concept probes on black boxes, such as TCAV by Been Kim and colleagues, measure a concept's influence after the fact.
• Feature engineering: Hand-computes input features from raw data. CBM concepts are predicted from raw inputs by a trained model and supervised with human labels.