Robotics

Robotics is the field of engineering that designs, builds, and programs robots: machines that sense their surroundings, decide what to do, and act on the physical world through motors and other actuators. It combines mechanical and electrical engineering, control theory, and computer science. Classic industrial robots replay pre-programmed motions inside fenced cells, and artificial intelligence (AI) lets newer ones cope with variation: finding parts that are never in exactly the same place, steering around people, and learning manipulation skills from demonstrations instead of hand-written code.

How a Robot Works: Sense, Plan, Act#

Most robot software follows a sense-plan-act structure that runs continuously:

• Sensing: Cameras, depth sensors, lidar, inertial measurement units (IMUs), joint encoders, and force-torque sensors measure the world and the robot itself.
• Perception: Computer vision models turn raw images into objects, people, surfaces, and distances through object detection, segmentation, pose estimation, and depth estimation.
• State estimation and mapping: Sensor fusion with filters such as the Kalman filter estimates the robot's position and motion, and visual SLAM builds a map of unknown spaces while tracking the robot within it.
• Planning: Task planners decide what to do next, and motion planners compute collision-free paths for wheels or arms.
• Control: Low-level controllers, from PID loops to model predictive control, drive the motors to follow the plan, often hundreds of times per second.

Middleware such as ROS 2 (Robot Operating System) connects these pieces as separate processes that exchange messages over named topics. The loop closes because every action changes what the sensors see next:

Vision in the Control Loop#

Perception has to deliver results fast enough for planning on the robot's own computer, which is often an embedded GPU module such as NVIDIA Jetson. The Ultralytics ROS quickstart runs Ultralytics YOLO26 as a ROS node that subscribes to a camera topic and publishes annotated detections and segmentations, detected class names, and object distances from depth images for other nodes to use. The same perception-to-action step fits in a few lines of Python, here halting a mobile robot the moment a person appears in its camera feed:

from ultralytics import YOLO

model = YOLO("yolo26n.pt")
video = "https://github.com/ultralytics/assets/releases/download/v0.0.0/decelera_portrait_min.mov"

# Sense-act loop: read frames as they arrive and stop as soon as a person is detected
for frame_idx, result in enumerate(model.predict(video, stream=True, classes=[0])):
    if len(result.boxes) > 0:
        print(f"Person detected in frame {frame_idx}: sending stop command")
        break

With stream=True, the model yields one result per frame as the video is read, the same pattern used with a live camera index such as source=0.

Learning-Based Robotics#

Learned perception is now standard, and learning increasingly reaches planning and control as well:

• Imitation learning: Robots learn from teleoperated demonstrations through behavioral cloning. Techniques such as action chunking and diffusion policies made this practical for fine manipulation, such as folding laundry or inserting connectors.
• Reinforcement learning in simulation: Policies trained with reinforcement learning over millions of simulated trials transfer to real hardware with the help of domain randomization, which varies physics and appearance so the real world looks like one more variation. Agile walking and running controllers for quadrupeds and humanoids are commonly trained this way.
• Robot foundation models: Vision-language-action models map camera images and a natural-language instruction directly to motor commands. Examples include Physical Intelligence's π0, Google DeepMind's Gemini Robotics, and NVIDIA's open Isaac GR00T models for humanoids. Training data is pooled across robots, as in the Open X-Embodiment dataset of over one million trajectories from 22 robot types.

These methods are often grouped under embodied AI and physical AI, the AI research fields concerned with agents that perceive and act through a body.

Robotics vs. Robotic Process Automation#

Robotic process automation (RPA) shares the word but has no hardware: RPA "bots" are software that automate repetitive digital work, such as copying data between business applications or processing invoices. Robotics always involves a physical machine, such as Boston Dynamics' Spot quadruped walking a construction site to capture inspection images.

Applications#

• Manufacturing: Articulated arms weld, paint, and assemble, while collaborative robots (cobots) work beside people at lower speeds and forces. Vision guides bin picking and inspects parts for defects.
• Logistics: Autonomous mobile robots (AMRs) navigate warehouses with onboard maps and perception, rerouting around obstacles, whereas older automated guided vehicles (AGVs) followed fixed tape or wire paths.
• Agriculture: Robots weed, thin, and harvest crops using vision to tell plants apart.
• Healthcare: Teleoperated surgical systems scale a surgeon's hand motions into precise instrument movements, and mobile robots deliver supplies in hospitals.
• Transportation: Autonomous vehicles and drones are mobile robots running the same sense-plan-act loop at higher speeds.

According to the International Federation of Robotics, 542,000 industrial robots were installed worldwide in 2024, 54% of them in China, bringing the number in operation to about 4.66 million.