Prompt Tuning

Prompt tuning adapts a pretrained foundation model to a new task by training a small set of vectors, called a soft prompt, that is placed in front of the input while every model weight stays frozen. The soft prompt lives in the same space as the model's token embeddings, but its vectors are learned by backpropagation and need not match any real word. Brian Lester, Rami Al-Rfou, and Noah Constant at Google introduced the method in The Power of Scale for Parameter-Efficient Prompt Tuning, presented at EMNLP 2021. They noted that a five-token prompt for the 11-billion-parameter T5-XXL model needs only 20,480 parameters per task, and showed that as models grow past billions of parameters, prompt tuning matches the accuracy of tuning the whole model.

How Prompt Tuning Works#

A large language model (LLM) turns each input token into an embedding vector before the transformer layers see it. Prompt tuning prepends k extra vectors to that sequence; Lester et al. tested lengths from 1 to 150 and used 100 by default. During training, the task's loss function is backpropagated through the frozen network all the way to the input, and the optimizer updates only the prompt vectors. The result is a small tensor per task. Because the model itself never changes, removing the prompt restores its original behavior with no risk of catastrophic forgetting, and one deployed copy can serve many tasks at once, with each request in a batch carrying its own soft prompt, which Lester et al. call mixed-task inference.

The following PyTorch example tunes an 8-vector soft prompt for sentiment analysis on a frozen GPT-2 from Hugging Face Transformers, with four training reviews and eight held-out ones:

import torch
from torch.nn.functional import cross_entropy
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2").requires_grad_(False)  # freeze all 124M weights
embed = model.get_input_embeddings()

# Soft prompt: 8 trainable vectors in GPT-2's embedding space, initialized from the word " review"
soft_prompt = torch.nn.Parameter(embed.weight[tokenizer.encode(" review")].repeat(8, 1).clone())
optimizer = torch.optim.Adam([soft_prompt], lr=0.01)
labels = tokenizer.encode(" positive negative")  # token ids for label 0 and label 1

# Reviews alternate between positive (label 0) and negative (label 1)
train = ["Loved every minute.", "A waste of time.", "Great story and acting.", "Boring and slow."]
test = ["What a wonderful film.", "I fell asleep halfway.", "A fantastic ending.", "Terrible dialogue."]
test += ["A joy to watch.", "Dull and predictable.", "Superb from start to end.", "I want my money back."]


def label_scores(text):
    """Score both label tokens after the soft prompt and the review."""
    ids = tokenizer.encode(f"{text} Sentiment:", return_tensors="pt")
    return model(inputs_embeds=torch.cat([soft_prompt[None], embed(ids)], dim=1)).logits[0, -1, labels]


def correct():
    """Count held-out reviews whose higher-scoring label is right."""
    with torch.no_grad():
        return sum(int(label_scores(text).argmax()) == i % 2 for i, text in enumerate(test))


print(f"Held-out reviews correct before tuning: {correct()}/8")
for step in range(30):
    loss = sum(cross_entropy(label_scores(text), torch.tensor(i % 2)) for i, text in enumerate(train))
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
print(f"After tuning: {correct()}/8, training {soft_prompt.numel():,} of {model.num_parameters():,} parameters")

It prints Held-out reviews correct before tuning: 5/8 and then After tuning: 7/8, training 6,144 of 124,439,808 parameters. Thirty steps of Adam on four examples changed only 6,144 numbers, 0.005% of the model, and the frozen GPT-2 classified two more unseen reviews correctly.

Prefix Tuning and Visual Prompt Tuning#

Prompt tuning belongs to a family of continuous-prompt methods within parameter-efficient fine-tuning (PEFT). Xiang Lisa Li and Percy Liang's prefix tuning learns vectors that are prepended at every transformer layer, about 0.1% of the parameters of GPT-2 or BART, and matched full fine-tuning on table-to-text generation and summarization. Prompt tuning simplifies this by learning vectors at the input layer only.

The idea carries over to vision. Visual Prompt Tuning (VPT) by Menglin Jia and colleagues prepends learnable tokens to the patch tokens of a frozen Vision Transformer (ViT), training less than 1% of the parameters, and outperformed full fine-tuning in many image classification settings. For vision-language models, Context Optimization (CoOp) by Kaiyang Zhou and colleagues replaces the hand-written words in a CLIP prompt such as "a photo of a" with learned vectors. Across 11 datasets, CoOp beat hand-crafted prompts with one or two labeled examples per class and improved on them by about 15% on average with 16 examples per class.

Open-vocabulary detectors adapt in a related way. Ultralytics YOLOE turns class names into text prompt embeddings, and its linear probing recipe for custom datasets fuses the embeddings of the dataset's class names into the final convolution of each classification branch, then trains only those layers while the backbone, neck, and rest of the head stay frozen.

Prompt Tuning vs. Related Methods#

• Prompt engineering: A person writes the prompt as text, sometimes called a hard prompt, and nothing is trained. Prompt tuning learns the prompt from labeled data and needs access to the model's gradients, so it applies to open-weight models.
• Full fine-tuning: Updates every weight, producing a full copy of the model per task and risking catastrophic forgetting of earlier skills. Lester et al. found prompt-tuned models were also more robust to domain shift than fully tuned ones.
• LoRA: Adds trainable low-rank matrices inside the model's weight layers. Prompt tuning leaves the layers untouched and changes only the input.
• Few-shot learning in the prompt: Puts labeled examples into the context window as text at every request. A tuned soft prompt stores what the examples teach in a few vectors and costs only k extra input positions.