Model Editing

Model editing is the targeted change of one specific fact or behavior in a trained model, such as the answer a large language model (LLM) gives to "The Space Needle is located in the city of", without retraining the model and without changing its outputs on unrelated inputs. It is mostly used to correct, update, or insert factual knowledge in LLMs, where full retraining is too expensive and plain fine-tuning on a single example tends to overfit or damage other knowledge. An editor either computes a small, localized change to the model weights or stores the edit outside the weights and routes relevant inputs to it. The best-known method, Rank-One Model Editing (ROME), was introduced by Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov in Locating and Editing Factual Associations in GPT, presented at NeurIPS 2022.

How Model Editing Works#

An edit request names an input, often a subject and a relation, and the new output, for example changing the Space Needle's city from Seattle to Paris. Editors fall into a few families:

• Locate-then-edit: ROME first uses causal tracing, an interpretability technique that corrupts and restores hidden states, to find where GPT models recall a fact. The decisive computation sits in the feed-forward (MLP) layers in the middle of the network, at the last token of the subject. ROME treats one MLP's output projection as a key-value memory and inserts the new fact with a closed-form rank-one weight update; in GPT-2 XL it edits layer 18 of 48. MEMIT spreads the update over several layers to insert thousands of facts at once.
• Learned editors: MEND trains a small hypernetwork that turns the gradient of the edit example into a low-rank weight update, which keeps edits fast for models with billions of parameters.
• Memory-based editors: SERAC stores edits in an explicit memory and uses a scope classifier to send inputs that concern an edit to a small counterfactual model, leaving the base weights untouched. GRACE inserts a codebook of discrete key-value adaptors at one layer for long sequences of edits. Putting the new fact in the prompt, as retrieval-augmented generation does, is the simplest editor of this kind.

ROME's update rests on a property of linear layers: a weight matrix W maps stored keys to values, and a new key k* can be given a new value v* by adding a rank-one matrix. ROME weights the update by the inverse covariance C of the keys the layer already sees, estimated from a sample of Wikipedia text, which points the change away from directions that existing keys use. This PyTorch example applies that update to a random layer that stores 5,000 keys and compares it with a plain rank-one update that ignores the covariance:

import torch

torch.manual_seed(0)
scales = torch.logspace(0, -2, 256)  # key dimensions with very different variances, as in real activations
keys = torch.randn(5000, 256) * scales  # keys of 5,000 facts the layer already stores
W = torch.randn(128, 256)  # the layer maps each key to a value
C = keys.T @ keys / len(keys)  # second-moment statistics of the stored keys

k_new = torch.randn(256) * scales  # key of the subject to edit
v_new = W @ keys[0]  # new value: one already stored for another subject, as in "Seattle" -> "Paris"

# Plain rank-one update vs. ROME's update, which steers the change away from directions stored keys use
plain = W + torch.outer(v_new - W @ k_new, k_new) / (k_new @ k_new)
c_inv_k = torch.linalg.solve(C, k_new)
rome = W + torch.outer(v_new - W @ k_new, c_inv_k) / (k_new @ c_inv_k)

for name, edited in [("Plain rank-one", plain), ("ROME", rome)]:
    error = (edited @ k_new - v_new).norm() / v_new.norm()
    drift = ((keys @ (edited - W).T).norm(dim=1) / (keys @ W.T).norm(dim=1)).median()
    print(f"{name}: new-fact error {error:.4f}, median change on stored facts {drift:.2%}")

Both updates map the new key exactly to its new value, with an error of 0.0000. The plain update changes the median output for the 5,000 stored keys by 10.99%, and ROME's covariance-weighted update by 5.02%, so the edit disturbs less of what the layer already knew.

Evaluating an Edit#

Edits are judged on several properties at once. Reliability asks whether the edited input now gives the new answer; generalization asks whether paraphrases do too; locality, also called specificity, asks whether unrelated inputs keep their old answers; and portability asks whether reasoning that depends on the fact follows it. ROME's authors built the CounterFact dataset of 21,919 counterfactual edits to measure these together, since earlier benchmarks let methods trade generalization for specificity unnoticed.

Edits often fail the harder tests. On the 5,000 edits of the RippleEdits benchmark, prominent editors failed to make consistent changes to facts logically connected to the edited one, and a simple in-context editing baseline scored best. Gupta et al. found that applying ROME and MEMIT sequentially makes a model forget earlier edits and downstream skills, first gradually and then abruptly, which limits how many edits a model can absorb.

Editing Vision Models#

The same rank-one machinery applies to images. Santurkar et al. edited image classifiers by rewriting the rule a layer applies to a visual concept, for example mapping the representation of snow to that of road so that the model handles vehicles on snowy roads, without collecting new training data. Ultralytics YOLOE makes a gradient-free edit of its own detection head on export: the classes configured with set_classes() are multiplied into the final convolution of each classification branch, so the exported model detects the new vocabulary with no training and no text encoder.

Model Editing vs. Related Concepts#

• Fine-tuning: Updates many weights from a dataset to change broad behavior. Editing targets one input-output pair and constrains everything else.
• Retrieval-augmented generation (RAG): Supplies current facts in the prompt at every request and leaves the model unchanged, which is easy to roll back and scales to many facts.
• Machine unlearning: Removes the influence of specific training data. Editing replaces or adds a fact, and an edit that deletes a fact is one form of unlearning.
• Task arithmetic: Adds or subtracts the weight changes of whole fine-tuned tasks. Model editing works at the level of a single fact.
• Mechanistic interpretability: Explains how a model computes. ROME's causal tracing comes from this field and decides which layer an edit targets.