Physical AI
Physical AI lets robots, vehicles, and other machines perceive the real world, reason about it, and act in it. Covers the stack, training, and uses.
Physical AI is artificial intelligence (AI) that perceives, reasons about, and acts in the physical world through a body: robots, drones, autonomous vehicles, and smart industrial machines. A chatbot's output is text on a screen, while a physical AI system's output is a motor command whose result depends on friction, gravity, collisions, and the people nearby, so the system must understand 3D space and cause and effect well enough to act safely in real time. NVIDIA made the term central to its robotics and simulation platforms, and it came into wide use as large pretrained models began to control real robots. It is a near-synonym of embodied AI, the older research term, with physical AI more common in industry.
How Physical AI Systems Work#
Every physical AI system closes a loop between sensing and acting, many times per second:
- Sensing: Cameras, lidar, radar, inertial measurement units, joint encoders, and force sensors measure the world and the machine's own state.
- Perception: Vision models turn raw pixels into objects, depth, poses, and maps that a planner can reason about.
- Decision-making: Planners, learned policies, or large robot foundation models choose the next action.
- Actuation: Motors, grippers, wheels, and brakes carry out the action, and the sensors measure its effect at the next step.
Sensor fusion combines the strengths of different sensors, computer vision does most of the perception work, and decisions come from controllers such as model predictive control, policies trained with reinforcement learning, or robot foundation models. Those foundation models increasingly split decision-making between a slower model that interprets instructions and the scene and a faster model that outputs motor commands, as in NVIDIA's open GR00T N1 humanoid model, which pairs a vision-language model with a separate action module:
Vision-language-action (VLA) models put both roles in one network. Google DeepMind's RT-2 showed in 2023 that a vision-language model fine-tuned to output robot actions as tokens carried knowledge from web data into control. Physical Intelligence's π0 adds a flow-matching action expert, close in spirit to diffusion policies, that outputs smooth action chunks of many future steps at once, and Google DeepMind's Gemini Robotics models extend the approach to dexterous multi-step tasks such as folding origami.
Perception to Action in Code#
Perception is the first link in the loop, and its speed bounds how fast the machine can react. Ultralytics YOLO26 is built for this edge role and runs on robot computers such as NVIDIA Jetson after TensorRT export. This example detects people in a camera frame and turns the most confident detection into a steering command with a proportional controller, the simplest form of visual servoing:
from ultralytics import YOLO
model = YOLO("yolo26n.pt")
# Perceive: detect people (COCO class 0) in the robot's camera frame
results = model("https://ultralytics.com/images/bus.jpg", classes=[0])
result = results[0]
# Act: steer toward the most confident person with a proportional controller
best = int(result.boxes.conf.argmax())
x_center = float(result.boxes.xywhn[best, 0]) # 0 = left edge of the frame, 1 = right edge
turn_rate = -1.5 * (x_center - 0.5) # rad/s; positive turns left, toward a person on the left
print(f"Person at {x_center:.2f} of frame width -> turn command {turn_rate:+.2f} rad/s")It prints where the chosen person sits across the frame and a turn command whose sign points the robot toward them and whose size grows with the offset. On a robot, the same two steps run on every camera frame, so the command shrinks to zero as the robot turns to face its target.
Training Physical AI#
Data is the central bottleneck: the internet holds trillions of words but very few recordings of robots acting, and each recording is tied to one robot's body and sensors. Teams collect teleoperation demonstrations, in which people drive the robot through a task while its sensors and commands are recorded, and train policies to imitate them. Physics simulators such as NVIDIA Isaac Lab run thousands of robots in parallel for deep reinforcement learning, with domain randomization of physics and appearance to narrow the gap between simulation and reality. Rendered scenes and digital twins supply labeled synthetic data of rare situations for perception models, and world models generate new variations of recorded scenes, such as the same street in rain or at night.
Real-World Applications#
- Logistics: Autonomous mobile robots navigate warehouses with maps they build themselves, rerouting around people and forklifts, while automated guided vehicles follow fixed paths.
- Manufacturing: Vision-guided arms pick randomly piled parts, assemble components, and inspect their own work, and humanoid robots are in pilot deployments moving totes and parts.
- Healthcare: In 2025, Johns Hopkins' SRT-H robot, trained on videos of surgeons, performed a key phase of gallbladder removal on eight ex vivo pig gallbladders without a human controlling its arms, while responding to spoken corrections.
- Agriculture: Harvesting and weeding robots identify individual plants with cameras and act on each one, such as removing a weed without spraying the whole field.
- Driving: Robotaxis and driver-assistance systems are the largest deployed physical AI fleets, perceiving traffic and controlling the vehicle many times per second.
Physical AI vs. Related Terms#
- Robotics: The engineering field of designing and building robots, including mechanics, electronics, and control. Physical AI is the learned intelligence that perceives and decides, and it also runs vehicles and machines that are not called robots.
- Generative AI: Produces text, images, video, and code. Its techniques now power physical AI too, from diffusion models that generate robot motions to world models that generate training scenes.
- Spatial intelligence: The ability to understand and reason about 3D space, objects, and their relations, a capability physical AI depends on.
- Edge AI: Running models on the device itself. Physical AI is almost always edge AI, because a robot cannot wait on a network round trip before it brakes.










