Latent Reasoning

Latent reasoning is reasoning that a model carries out in its hidden states, the continuous vectors inside the neural network, instead of in written tokens. With chain-of-thought prompting, a large language model (LLM) writes each intermediate step as text and reads it back; with latent reasoning, the intermediate steps stay as vectors and only the answer is decoded. The term covers two things: the implicit reasoning that happens inside any single forward pass, such as resolving a two-step question without naming the connecting fact, and methods that train models to think in continuous space, such as Meta's Coconut, published in December 2024, and recurrent-depth models from 2025.

Why Reason in Latent Space#

A written step passes forward very little information. Each token is one choice from a vocabulary of tens of thousands, while a hidden state in even a small transformer holds hundreds or thousands of numbers that can encode several partial results at once. Writing steps out also costs time: every reasoning token is a full forward pass, so a long chain of thought adds inference latency and serving cost. Extra hidden computation helps even without words. Researchers at Google and Carnegie Mellon University found in 2023 that training a 1-billion-parameter model with learnable "pause" tokens, which give it extra forward passes before answering, produced an 18% gain in exact-match score on the SQuAD question answering benchmark.

Coconut: Chain of Continuous Thought#

Researchers at Meta and UC San Diego introduced Coconut in Training Large Language Models to Reason in a Continuous Latent Space. Between special begin- and end-of-thought tokens, the model feeds its last hidden state straight back in as the next input embedding instead of decoding it into a word. Coconut used OpenAI's GPT-2, and the same loop runs on the pretrained model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")
ids = tokenizer("Question: 3 apples plus 4 apples is", return_tensors="pt").input_ids
embeds = model.get_input_embeddings()(ids)

# Continuous thoughts: the last hidden state becomes the next input embedding
with torch.no_grad():
    for _ in range(4):
        hidden = model(inputs_embeds=embeds, output_hidden_states=True).hidden_states[-1]
        thought = hidden[:, -1:]  # a vector that is never decoded into a token
        embeds = torch.cat([embeds, thought], dim=1)

print(f"Positions: {ids.shape[1]} prompt tokens + 4 thoughts = {embeds.shape[1]}")
print(f"Each thought: {thought.shape[-1]} floats; each written step: 1 of {tokenizer.vocab_size:,} tokens")

Four reasoning steps grow the sequence from 8 to 12 positions without producing any text, and each step hands the next forward pass 768 floating-point numbers, where a written step would hand over one of 50,257 tokens, about 15.6 bits of information. Coconut fine-tunes GPT-2 through this loop with a curriculum that replaces written reasoning steps with continuous thoughts one stage at a time. On ProsQA, a logical reasoning task that requires planning, it reached 97.0% accuracy while generating 14.2 tokens on average, against 77.5% and 49.4 tokens for chain-of-thought. On GSM8K math problems, it trailed chain-of-thought, 34.1% to 42.9%, while generating about a third as many tokens. The authors found that a single continuous thought can encode several possible next steps at once, letting the model explore alternatives in a breadth-first way that a written chain, committed to one token at a time, cannot.

Other Approaches#

Recurrent-depth models reuse the same block of layers many times. Jonas Geiping and colleagues trained a 3.5-billion-parameter model on 800 billion tokens in 2025 that loops a recurrent block at inference, and its reasoning benchmark scores kept rising with more iterations, up to compute equivalent to a 50-billion-parameter model, without writing a single extra token. That makes recurrence a way to scale test-time compute, an axis of scaling laws usually reached by generating longer chains of thought. Implicit chain-of-thought methods train a model on worked solutions and then remove the written steps gradually: Yuntian Deng, Yejin Choi, and Stuart Shieber's stepwise internalization taught GPT-2 Small to solve 9-by-9 digit multiplication with up to 99% accuracy without intermediate steps.

The idea extends beyond text. Meta's large concept models (LCMs) predict the next sentence embedding instead of the next token, joint embedding predictive architectures (JEPA) predict the missing parts of images and video in latent space rather than in pixels, and embodied AI systems such as Figure's Helix pass a latent vector, not text, from a vision-language model (VLM) to the policy that moves the robot. Detectors hold the same kind of hidden state: in Ultralytics YOLO26, model.embed() runs predict mode only up to the layer before the detection head and average-pools its feature map into one vector per image, in place of boxes and class labels.

Interpretability and Safety#

A written chain of thought gives humans and automated monitors a record to read, and reasoning models that reason in text can be checked for signs of reward hacking or deception before they act. A July 2025 position paper by researchers from OpenAI, Google DeepMind, Anthropic, Meta, and other groups called this chain-of-thought monitorability a fragile opportunity for AI safety and warned that latent reasoning architectures could remove it. Latent steps can be read only indirectly, with mechanistic interpretability tools such as probes, the logit lens, and sparse autoencoders (SAEs); in the Coconut paper, decoding a continuous thought through the language model head spread its probability over several candidate intermediate values.