Reasoning Models
Reasoning models are LLMs trained with reinforcement learning to write a long chain of thought before answering. Covers o1, DeepSeek-R1, and limits.
A reasoning model is a large language model (LLM) trained to generate a long internal chain of thought before its final answer, so it can spend more computation on harder problems. OpenAI introduced the category with o1 on September 12, 2024: on the 2024 AIME math competition, GPT-4o solved 12% of problems on average, while o1 solved 74% with one sample per problem, 83% with a consensus vote over 64 samples, and 93% when a learned scoring function reranked 1,000 samples. o1 also surpassed the accuracy of human PhD experts on the GPQA science benchmark. DeepSeek-R1, Google's Gemini thinking models, Anthropic's Claude with extended thinking, and Alibaba's Qwen3 followed, and many current models are hybrids that answer quickly or think at length depending on a setting.
How Reasoning Models Are Trained#
Chain-of-thought prompting showed in 2022 that asking a model to write intermediate steps improves its answers. Reasoning models learn that behavior through reinforcement learning (RL) on problems whose answers can be checked automatically, an approach called reinforcement learning with verifiable rewards (RLVR). DeepSeek described the recipe in its DeepSeek-R1 paper in January 2025. Its R1-Zero model started from the DeepSeek-V3 base model and trained with Group Relative Policy Optimization (GRPO), rewarded only for a correct final answer and for the right output format:
No step-by-step examples were provided, yet R1-Zero's pass@1 accuracy on AIME 2024 rose from 15.6% to 71.0%, and to 86.7% with majority voting. Its responses grew longer during training, and the model began to re-check and revise its own steps, a moment the authors called an "aha moment." The released DeepSeek-R1 first trained on a few thousand curated long reasoning examples for readability and reached 79.8% on AIME 2024, and DeepSeek distilled its reasoning traces into smaller Qwen and Llama models. A process reward model (PRM) that scores each step is the main alternative to these outcome rewards, though DeepSeek reported that PRMs and Monte Carlo tree search did not pay off in its large-scale training.
Test-Time Compute#
A reasoning model's accuracy rises with the number of tokens it spends thinking, a second axis of scaling laws beside model size and training data. APIs expose this as a reasoning effort or thinking budget, and accuracy can rise further by sampling several answers in parallel and voting or by searching over partial solutions with a verifier. The cost is latency and tokens: a hard problem can consume thousands of hidden reasoning tokens before the first word of the answer. The Ultralytics LLM interface forwards request arguments unchanged to the OpenAI SDK, so a YOLO26 pipeline can pass reasoning={"effort": "high"} to a reasoning model through the Responses API when a detected scene needs a deliberate answer.
OpenAI's o3 and o4-mini, released in April 2025, extended the chain of thought to images, cropping, zooming, and rotating a photo while reasoning about it. This visual reasoning helps with visual question answering (VQA) on small text, charts, and cluttered scenes.
Limits#
Reasoning models overthink simple questions, spending tokens where a direct answer would do, and they still fail unpredictably on some easy tasks, a pattern known as jagged intelligence. RL on checkable answers rewards outcomes, so a model can learn shortcuts that pass the checker, a form of reward hacking. The written chain of thought is also an imperfect record of the computation: in an Anthropic study published in April 2025, models given a hint that changed their answer mentioned the hint only 25% of the time for Claude 3.7 Sonnet and 39% of the time for DeepSeek-R1.
Reasoning Models vs. Related Terms#
- Chain-of-thought prompting: Prompts any LLM to write out its steps at inference time, the behavior reasoning models are trained to produce unprompted.
- System 2 thinking: The psychology term for slow, deliberate reasoning that the field borrows to describe what reasoning models do.
- Latent reasoning: Reasoning carried out in hidden-state vectors instead of written tokens, which leaves no readable trace.
- Tree of thoughts: An explicit search over branching chains of thought, guided by the model's own evaluation of each branch.
