Steering Vectors

A steering vector is a direction in a neural network's activation space that corresponds to a concept or behavior, such as positive sentiment, refusal, or flattery. Adding a scaled copy of the vector to the network's hidden activations during the forward pass pushes the output toward that behavior, and subtracting it pushes the output away, while the model weights stay unchanged. The technique belongs to activation engineering and is mostly applied to large language models (LLMs), where it relies on the observation that many high-level concepts are represented as roughly linear directions in the residual stream, the running sum of activations that each transformer layer reads from and writes to.

How Steering Vectors Are Built and Applied#

The most common recipe is a difference of means. Researchers run pairs of contrasting inputs through the model, one that shows the behavior and one that does not, record the activations at a chosen layer, and average the differences. Activation Addition (ActAdd), introduced by Alexander Matt Turner and colleagues in 2023, showed that a single pair such as "Love" and "Hate" can steer the sentiment of GPT-2-XL. Contrastive Activation Addition (CAA) averages over many multiple-choice pairs to steer Llama 2 Chat toward or away from behaviors such as sycophancy, hallucination, and refusal, adding the vector at every token position after the user's prompt. Directions can also come from linear probes, from principal components of the differences, or from the features of a sparse autoencoder.

At inference, a hook adds the coefficient times the vector to the chosen layer's output. Intermediate layers usually steer best, and the coefficient sets the strength: too small has no visible effect, and too large degrades the text into repetition or nonsense.

The ActAdd experiment fits in a few lines of PyTorch with the 124-million-parameter GPT-2 from Hugging Face Transformers. A forward hook adds the "Love" minus "Hate" vector to the output of block 6 while the model completes a hostile prompt:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")
layer = 6


def activation(text):
    """Return the last token's activation at the output of the steering layer."""
    with torch.no_grad():
        outputs = model(**tokenizer(text, return_tensors="pt"), output_hidden_states=True)
    return outputs.hidden_states[layer + 1][0, -1]


# Steering vector: activation difference between a contrastive pair of prompts
steering_vector = activation(" Love") - activation(" Hate")
prompt = tokenizer("I hate you because", return_tensors="pt")

for coeff in [0.0, 1.0]:
    # Add the scaled vector to the layer's output at every position while generating
    hook = model.transformer.h[layer].register_forward_hook(
        lambda module, args, output: output + coeff * steering_vector
    )
    tokens = model.generate(**prompt, max_new_tokens=16, do_sample=False, pad_token_id=tokenizer.eos_token_id)
    hook.remove()
    print(f"coeff={coeff}: {tokenizer.decode(tokens[0])}")

Unsteered, the model continues "I hate you because I hate you. I hate you because I hate you." With the vector added at coefficient 1.0, the same greedy decoding continues "I hate you because you are a good person. I love you. I love you." The weights are identical in both runs; only the activations of one layer changed.

Steering Vectors vs. Related Concepts#

• Task vectors: The difference between fine-tuned and original weights, added to a model to transfer a skill. Task vectors edit weights once, and steering vectors edit activations on every forward pass.
• Representation engineering (RepE): The broader approach of reading and controlling a model through its internal representations. Steering vectors are its main control tool.
• Prompt engineering: Guides behavior through the input text. Steering acts on internal activations, so it needs access to the model's internals and works only with open-weight models or the provider's own systems.
• Fine-tuning: Changes behavior permanently by training the weights with gradient descent. Building a steering vector needs only forward passes on a handful to a few hundred examples.
• Mechanistic interpretability: Explains how models compute. Steering is one of its tools for testing whether a direction it found actually causes a behavior.

Real-World Applications#

Steering vectors are a research tool for AI safety as much as a control method. In 13 open chat models up to 72 billion parameters, refusal is mediated by a single direction: erasing it stops the model from refusing harmful requests, and adding it makes the model refuse harmless ones, which shows how easily weight-level safety training can be undone in open models.

Truth-related directions reduce hallucinations. Inference-Time Intervention shifts the activations of a few attention heads along such directions, raising the Alpaca model's truthfulness on the TruthfulQA benchmark from 32.5% to 65.1%. Anthropic's persona vectors apply the same idea to character traits such as sycophancy, a propensity to hallucinate, and malicious intent, monitoring them in deployment, flagging training data that would strengthen them, and steering against them during fine-tuning. In reasoning models, vectors extracted from DeepSeek-R1-Distill models control reasoning behaviors such as expressing uncertainty, testing examples, and backtracking.

Vision models encode concepts as directions too, and the same recipe applies to the feature maps of a computer vision network. In Ultralytics YOLO26, model.embed() returns each image's average-pooled activations from any chosen layer, which gives the per-image activations needed to compute a mean-difference direction between two sets of images, such as daytime and nighttime scenes.

Limitations#

Steering is less reliable than its best demonstrations suggest. A study of the generalization and reliability of steering vectors by Daniel Tan and colleagues found that steerability varies widely from input to input, that spurious features of the contrastive prompts can drive the effect, and that small, reasonable changes to a prompt can make a vector stop working. Steering also costs fluency: in the ActAdd paper's sentiment experiments, the perplexity-based disfluency of steered text was 1.5 to 3 times that of unsteered text.