License YOLO
Back to Ultralytics Glossary

Physical AI

Physical AI lets robots, vehicles, and other machines perceive the real world, reason about it, and act in it. Covers the stack, training, and uses.

Physical AI is artificial intelligence (AI) that perceives, reasons about, and acts in the physical world through a body: robots, drones, autonomous vehicles, and smart industrial machines. A chatbot's output is text on a screen, while a physical AI system's output is a motor command whose result depends on friction, gravity, collisions, and the people nearby, so the system must understand 3D space and cause and effect well enough to act safely in real time. NVIDIA made the term central to its robotics and simulation platforms, and it came into wide use as large pretrained models began to control real robots. It is a near-synonym of embodied AI, the older research term, with physical AI more common in industry.

How Physical AI Systems Work#

Every physical AI system closes a loop between sensing and acting, many times per second:

  • Sensing: Cameras, lidar, radar, inertial measurement units, joint encoders, and force sensors measure the world and the machine's own state.
  • Perception: Vision models turn raw pixels into objects, depth, poses, and maps that a planner can reason about.
  • Decision-making: Planners, learned policies, or large robot foundation models choose the next action.
  • Actuation: Motors, grippers, wheels, and brakes carry out the action, and the sensors measure its effect at the next step.

Sensor fusion combines the strengths of different sensors, computer vision does most of the perception work, and decisions come from controllers such as model predictive control, policies trained with reinforcement learning, or robot foundation models. Those foundation models increasingly split decision-making between a slower model that interprets instructions and the scene and a faster model that outputs motor commands, as in NVIDIA's open GR00T N1 humanoid model, which pairs a vision-language model with a separate action module:

Vision-language-action (VLA) models put both roles in one network. Google DeepMind's RT-2 showed in 2023 that a vision-language model fine-tuned to output robot actions as tokens carried knowledge from web data into control. Physical Intelligence's π0 adds a flow-matching action expert, close in spirit to diffusion policies, that outputs smooth action chunks of many future steps at once, and Google DeepMind's Gemini Robotics models extend the approach to dexterous multi-step tasks such as folding origami.

Perception to Action in Code#

Perception is the first link in the loop, and its speed bounds how fast the machine can react. Ultralytics YOLO26 is built for this edge role and runs on robot computers such as NVIDIA Jetson after TensorRT export. This example detects people in a camera frame and turns the most confident detection into a steering command with a proportional controller, the simplest form of visual servoing:

from ultralytics import YOLO

model = YOLO("yolo26n.pt")

# Perceive: detect people (COCO class 0) in the robot's camera frame
results = model("https://ultralytics.com/images/bus.jpg", classes=[0])
result = results[0]

# Act: steer toward the most confident person with a proportional controller
best = int(result.boxes.conf.argmax())
x_center = float(result.boxes.xywhn[best, 0])  # 0 = left edge of the frame, 1 = right edge
turn_rate = -1.5 * (x_center - 0.5)  # rad/s; positive turns left, toward a person on the left
print(f"Person at {x_center:.2f} of frame width -> turn command {turn_rate:+.2f} rad/s")

It prints where the chosen person sits across the frame and a turn command whose sign points the robot toward them and whose size grows with the offset. On a robot, the same two steps run on every camera frame, so the command shrinks to zero as the robot turns to face its target.

Training Physical AI#

Data is the central bottleneck: the internet holds trillions of words but very few recordings of robots acting, and each recording is tied to one robot's body and sensors. Teams collect teleoperation demonstrations, in which people drive the robot through a task while its sensors and commands are recorded, and train policies to imitate them. Physics simulators such as NVIDIA Isaac Lab run thousands of robots in parallel for deep reinforcement learning, with domain randomization of physics and appearance to narrow the gap between simulation and reality. Rendered scenes and digital twins supply labeled synthetic data of rare situations for perception models, and world models generate new variations of recorded scenes, such as the same street in rain or at night.

Real-World Applications#

  • Logistics: Autonomous mobile robots navigate warehouses with maps they build themselves, rerouting around people and forklifts, while automated guided vehicles follow fixed paths.
  • Manufacturing: Vision-guided arms pick randomly piled parts, assemble components, and inspect their own work, and humanoid robots are in pilot deployments moving totes and parts.
  • Healthcare: In 2025, Johns Hopkins' SRT-H robot, trained on videos of surgeons, performed a key phase of gallbladder removal on eight ex vivo pig gallbladders without a human controlling its arms, while responding to spoken corrections.
  • Agriculture: Harvesting and weeding robots identify individual plants with cameras and act on each one, such as removing a weed without spraying the whole field.
  • Driving: Robotaxis and driver-assistance systems are the largest deployed physical AI fleets, perceiving traffic and controlling the vehicle many times per second.
  • Robotics: The engineering field of designing and building robots, including mechanics, electronics, and control. Physical AI is the learned intelligence that perceives and decides, and it also runs vehicles and machines that are not called robots.
  • Generative AI: Produces text, images, video, and code. Its techniques now power physical AI too, from diffusion models that generate robot motions to world models that generate training scenes.
  • Spatial intelligence: The ability to understand and reason about 3D space, objects, and their relations, a capability physical AI depends on.
  • Edge AI: Running models on the device itself. Physical AI is almost always edge AI, because a robot cannot wait on a network round trip before it brakes.

Explore solutions

Computer vision in sports

Computer vision in sports

Discover how Ultralytics YOLO powers sports vision with player and ball tracking, performance analytics, coaching insights, and athlete rehabilitation.
Learn more
Computer vision for aerial imagery

Computer vision for aerial imagery

Turn drone and aerial imagery into real-time insights for agriculture, aquaculture, conservation, and geospatial analysis with Ultralytics YOLO.
Learn more
Computer vision in aerospace

Computer vision in aerospace

Apply computer vision in aerospace with Ultralytics YOLO for aircraft inspection, runway FOD detection, airport ground operations, and composite defect detection.
Learn more
Computer vision in security

Computer vision in security

Protect every site with Ultralytics YOLO models. Vision AI powers perimeter monitoring, threat detection, license-plate recognition, and workplace safety.
Learn more
Computer vision in robotics

Computer vision in robotics

Power smarter machines with Ultralytics YOLO models. Vision AI in robotics drives autonomous navigation, perception, object tracking, and real-time control.
Learn more
Computer vision in logistics

Computer vision in logistics

Streamline logistics with Ultralytics YOLO models. Vision AI enables package inspection, sorting, vehicle tracking, and real-time warehouse safety monitoring.
Learn more
Computer vision in retail

Computer vision in retail

Reimagine retail with Ultralytics YOLO models. Vision AI powers inventory tracking, shelf monitoring, queue management, and smarter customer insights.
Learn more
Computer vision in healthcare

Computer vision in healthcare

Build healthcare solutions with Ultralytics YOLO models. Vision AI in healthcare powers faster medical imaging, smarter diagnostics, and patient monitoring.
Learn more
Computer vision in manufacturing

Computer vision in manufacturing

Optimize manufacturing with Ultralytics YOLO models. Vision AI drives quality control, defect detection, PPE compliance, and assembly line automation.
Learn more
Computer vision in automotive

Computer vision in automotive

Apply computer vision in automotive with Ultralytics YOLO models. Vision AI elevates road safety, driver assistance, and vehicle automation for smarter roads.
Learn more
Computer vision in agriculture

Computer vision in agriculture

Bring vision AI to smart agriculture with Ultralytics YOLO models. Power crop monitoring, livestock tracking, and precision farming for higher, smarter yields.
Learn more

Let's build the future of AI together!

Begin your journey with the future of machine learning