Spatial Intelligence

Spatial intelligence in AI is a system's ability to perceive the three-dimensional structure of the world, reason about where things are and how they relate to each other, and use that understanding to act, answer questions, or generate new scenes. A model with spatial intelligence knows that a cup is on a table and within reach, that a hallway continues around a corner, and how a scene will look from another viewpoint. The term comes from psychology, where spatial intelligence is one of the abilities in Howard Gardner's theory of multiple intelligences (1983). In AI it was popularized by Fei-Fei Li, whose 2025 essay From Words to Worlds called it the next frontier after language, and it underpins robotics, augmented reality, and autonomous driving.

Building Blocks of Spatial Intelligence#

Spatial understanding combines several capabilities:

• Depth and 3D geometry: Depth estimation, stereo vision, and 3D object detection recover distances and object extents from images. Feed-forward reconstruction models such as Meta and Oxford's VGGT, the CVPR 2025 best paper, predict camera poses, depth maps, and 3D point maps from one to hundreds of images in seconds.
• Localization and mapping: Visual SLAM tracks a moving camera while building a map of its surroundings.
• Orientation and articulation: Pose estimation recovers how people and objects are posed, and oriented bounding boxes capture the rotation of objects seen from above, as in aerial images.
• 3D representations: Point clouds, meshes, neural radiance fields, and Gaussian splatting store the reconstructed scene for rendering, simulation, and planning.
• Spatial reasoning: Relations such as above, behind, and nearer than, plus distances, sizes, and routes, expressed in language or used directly for decisions.

Different tasks draw on the same scene model in different ways:

From 2D Detections to 3D Positions#

A detector reports where objects appear in an image; adding depth tells a system how far away they are. Ultralytics YOLO26 includes a monocular depth estimation task whose models predict a per-pixel depth map in meters, so pairing a YOLO26 detector with a YOLO26 depth model places every detected object at a distance from the camera:

from ultralytics import YOLO

detector = YOLO("yolo26n.pt")
depth_model = YOLO("yolo26n-depth.pt")
image = "https://ultralytics.com/images/bus.jpg"

boxes = detector(image)[0].boxes
depth = depth_model(image)[0].depth.data  # per-pixel depth in meters, shape (H, W)

# Read the depth at each box center to place the objects in 3D
for cls, (x, y, w, h) in zip(boxes.cls.tolist(), boxes.xywh.tolist()):
    print(f"{detector.names[int(cls)]}: {depth[int(y), int(x)]:.1f} m")

With the camera's intrinsics, each box center and its depth convert to x, y, z coordinates in meters, which is the kind of 3D layout a grasp planner or a spatial question answerer needs.

Spatial Intelligence vs. Related Terms#

• Computer vision: The broader field of extracting information from images. Much of it works in the 2D image plane, classifying images and boxing objects, and spatial intelligence builds on it by adding metric 3D structure and relations.
• Spatial computing: The human-computer interaction paradigm in which digital content is placed in and responds to physical space, as in mixed-reality headsets. It is an application that depends on spatial intelligence.
• Embodied AI: Agents that perceive and act through a physical or simulated body. Spatial intelligence is one of the capabilities such an agent needs.
• World models: Models that predict how an environment will change. Spatially grounded world models generate or simulate consistent 3D scenes.

Spatial Reasoning in Multimodal Models#

Vision-language models describe images fluently but have lagged at spatial reasoning. VSI-Bench, a benchmark of over 5,000 questions about videos of indoor scenes, found in 2024 that multimodal LLMs showed competitive but subhuman visual-spatial intelligence, with spatial reasoning as the main bottleneck; prompting the models to draw an explicit cognitive map of the room before answering improved their distance judgments. The Ultralytics VLM Leaderboard measures a narrower spatial skill, how accurately vision language models place boxes around objects in real-world images. Generative models are attacking the problem from the other direction. World Labs, the company Fei-Fei Li cofounded, released Marble in November 2025, a world model that turns text, images, or video into persistent, navigable 3D environments exportable as Gaussian splats or meshes.