License YOLO
Back to Ultralytics Glossary

Instruction Tuning

Instruction tuning fine-tunes a pretrained language model on instruction-response pairs so it follows requests. Covers FLAN, InstructGPT, data, and VLMs.

Instruction tuning is fine-tuning a pretrained large language model (LLM) on pairs of instructions and desired responses so that it follows requests written in plain language. A pretrained model only continues text: asked by OpenAI to "Explain the moon landing to a 6 year old in a few sentences," GPT-3 continued with similar requests about gravity and relativity, while the instruction-tuned InstructGPT wrote the explanation. Google researchers described the method in the 2021 FLAN paper, which tuned a 137-billion-parameter model on more than 60 natural language processing (NLP) datasets rewritten as instructions; the tuned model beat the zero-shot performance of the 175-billion-parameter GPT-3 on 20 of 25 tasks it had never seen. In chat-model pipelines, the same step is usually called supervised fine-tuning (SFT), and the two terms are used as synonyms.

How Instruction Tuning Works#

Each example is formatted with a template that marks where the instruction ends and the response begins, such as the chat templates of current models or the "### Instruction" and "### Response" headers used by Stanford's Alpaca. Training uses the same next-token loss function as pretraining, but only the response tokens count, so the model learns to produce answers instead of to predict the instructions themselves. One training example on OpenAI's GPT-2 shows the setup:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")
optimizer = torch.optim.AdamW(model.parameters(), lr=5e-5)

# One instruction-response pair in the Alpaca prompt format
prompt_ids = tokenizer("### Instruction:\nName three animals that can fly.\n\n### Response:\n").input_ids
response_ids = tokenizer("Bats, eagles, and bees." + tokenizer.eos_token).input_ids
input_ids = torch.tensor([prompt_ids + response_ids])

# Label -100 drops the instruction from the loss, so the model learns only to respond
labels = torch.tensor([[-100] * len(prompt_ids) + response_ids])

for step in range(3):
    loss = model(input_ids=input_ids, labels=labels).loss
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()
    print(f"step {step}: loss {loss.item():.2f}")

Only the 10 response tokens count toward the loss, and three updates on this single example cut it from 4.45 to 1.49. An instruction-tuning run applies the same update across thousands to millions of different instruction-response pairs, so the model learns the general habit of answering instead of one answer.

Instruction tuning is usually the first stage of post-training, and the data can come from several sources:

Instruction Datasets#

Google's Flan collection grew to 1,836 tasks in 2022, and instruction tuning on it improved the 540-billion-parameter PaLM by 9.4% on average. OpenAI's InstructGPT combined labeler-written demonstrations with reinforcement learning from human feedback (RLHF), and people preferred the outputs of its 1.3-billion-parameter model to those of the 175-billion-parameter GPT-3. Stanford's Alpaca, released in March 2023, fine-tuned Meta's LLaMA 7B for less than $600 on 52,000 examples that an OpenAI model generated with the Self-Instruct method, an early example of synthetic data as a source of instruction data. Meta's LIMA showed how far a small dataset goes: it tuned a 65-billion-parameter LLaMA on only 1,000 carefully chosen examples, and its authors concluded that a model learns almost all of its knowledge during pretraining while instruction tuning teaches the format and style of answers. Methods such as direct preference optimization (DPO) then refine the tuned model with preference data.

Visual Instruction Tuning#

Vision-language models (VLMs) learn to follow instructions about images the same way. LLaVA, published in April 2023, gave text-only GPT-4 the captions and bounding boxes of COCO images and had it write 158,000 conversations, detailed descriptions, and complex reasoning examples about them, then tuned a model that connects a CLIP image encoder to a language model on that data, a method known as visual instruction tuning. Instruction-tuned VLMs answer free-form requests about a photo, and the LLM block in Ultralytics Platform Agents relies on that ability: it sends the image, the upstream YOLO26 results, and a written prompt to the selected vision-language model.

Prompt tuning learns a few soft prompt vectors and leaves the model frozen, and prompt engineering changes only the input text, while instruction tuning updates the model's weights. Domain fine-tuning adapts a model to a field such as medicine or law without necessarily teaching it to follow requests. Instruction tuning can also run with parameter-efficient fine-tuning (PEFT): QLoRA trains LoRA adapters on a 4-bit quantized model, and its authors instruction-tuned a 65-billion-parameter model on a single 48 GB GPU.

Explore solutions

Computer vision in sports

Computer vision in sports

Discover how Ultralytics YOLO powers sports vision with player and ball tracking, performance analytics, coaching insights, and athlete rehabilitation.
Learn more
Computer vision for aerial imagery

Computer vision for aerial imagery

Turn drone and aerial imagery into real-time insights for agriculture, aquaculture, conservation, and geospatial analysis with Ultralytics YOLO.
Learn more
Computer vision in aerospace

Computer vision in aerospace

Apply computer vision in aerospace with Ultralytics YOLO for aircraft inspection, runway FOD detection, airport ground operations, and composite defect detection.
Learn more
Computer vision in security

Computer vision in security

Protect every site with Ultralytics YOLO models. Vision AI powers perimeter monitoring, threat detection, license-plate recognition, and workplace safety.
Learn more
Computer vision in robotics

Computer vision in robotics

Power smarter machines with Ultralytics YOLO models. Vision AI in robotics drives autonomous navigation, perception, object tracking, and real-time control.
Learn more
Computer vision in logistics

Computer vision in logistics

Streamline logistics with Ultralytics YOLO models. Vision AI enables package inspection, sorting, vehicle tracking, and real-time warehouse safety monitoring.
Learn more
Computer vision in retail

Computer vision in retail

Reimagine retail with Ultralytics YOLO models. Vision AI powers inventory tracking, shelf monitoring, queue management, and smarter customer insights.
Learn more
Computer vision in healthcare

Computer vision in healthcare

Build healthcare solutions with Ultralytics YOLO models. Vision AI in healthcare powers faster medical imaging, smarter diagnostics, and patient monitoring.
Learn more
Computer vision in manufacturing

Computer vision in manufacturing

Optimize manufacturing with Ultralytics YOLO models. Vision AI drives quality control, defect detection, PPE compliance, and assembly line automation.
Learn more
Computer vision in automotive

Computer vision in automotive

Apply computer vision in automotive with Ultralytics YOLO models. Vision AI elevates road safety, driver assistance, and vehicle automation for smarter roads.
Learn more
Computer vision in agriculture

Computer vision in agriculture

Bring vision AI to smart agriculture with Ultralytics YOLO models. Power crop monitoring, livestock tracking, and precision farming for higher, smarter yields.
Learn more

Let's build the future of AI together!

Begin your journey with the future of machine learning