Edge Computing

Edge computing is the practice of running computation and storing data close to where data is produced or used, on devices, on-premises servers and gateways, or small data centers at the edge of telecom networks, instead of sending everything to a distant cloud region. Processing near the source cuts response time, reduces the bandwidth needed to move raw data, keeps sensitive data on site, and lets systems keep working when the network drops. The idea grew from content delivery networks that cached web pages near users in the late 1990s; researchers led by Mahadev Satyanarayanan proposed nearby "cloudlets" for mobile devices in 2009, Cisco introduced the term fog computing in 2012, and the European standards body ETSI began standardizing Multi-access Edge Computing for mobile networks in 2014. Running machine learning models on this infrastructure is called edge AI.

Edge Tiers#

"The edge" covers several layers between a sensor and the cloud:

• Device edge: Cameras, phones, drones, robots, and vehicles that process data on their own processors, often with built-in accelerators.
• On-premises edge: Gateways and compact servers inside a factory, store, hospital, or farm that aggregate data from many devices, such as one GPU box analyzing dozens of camera streams.
• Network edge: Servers in telecom base stations or regional points of presence, a few milliseconds from mobile users, which run workloads for devices too small to run them locally.
• Cloud: Central regions that train models, store long-term data, and manage the fleet of edge devices.

Why Process Data at the Edge#

Latency is the first reason. Light travels through optical fiber at about 200 kilometers per millisecond, so a data center 1,000 km away adds at least 10 ms of round-trip delay before any routing, queuing, or processing, and autonomous vehicles and robots that must react within milliseconds run real-time inference on board. Bandwidth is the second: a single 1080p H.264 camera stream needs a few megabits per second, so a site with 100 cameras would upload hundreds of megabits continuously, while a detection event sent as JSON takes a few kilobytes. Processing on site also keeps raw images of people local, so only counts, alerts, or anonymized results leave the building, which simplifies data privacy obligations, and it keeps systems running through network outages on farms, ships, mines, and remote infrastructure.

In practice, the edge and the cloud form a loop. Devices handle every frame locally and send only events and selected samples upward, and the cloud uses those samples to retrain and push improved models back out:

This pattern runs quality inspection on production lines, where a defective part must be rejected before the next one arrives, crop and livestock monitoring in fields with weak connectivity, and traffic management systems that adjust signals from cameras at intersections.

Edge Computing vs. Related Terms#

Cloud computing centralizes elastic capacity in large data centers, and edge computing extends computation outward from it, so most systems use both. Edge AI is the subset of edge computing that runs AI models. Fog computing, Cisco's term, describes the layer of gateways and local servers between devices and the cloud and is largely a synonym for the on-premises and network tiers. The Internet of Things (IoT) is the network of connected sensors and devices, and edge computing is where much of the data they produce gets processed.

Deploying Models to Edge Devices#

Edge hardware has a fraction of a data center GPU's memory, compute, and power budget, from microcontrollers drawing milliwatts to NVIDIA Jetson modules and Raspberry Pi boards. Models are made smaller and faster with model quantization to 8-bit integers, pruning, and knowledge distillation, then compiled for the target's runtime and accelerator, such as TensorRT on NVIDIA GPUs, OpenVINO on Intel CPUs and NPUs, LiteRT on Android, or Core ML on Apple devices. The export mode of Ultralytics YOLO produces these formats from one trained model. This example exports YOLO26 to NCNN, Tencent's inference framework for mobile and ARM CPUs, and runs the exported model:

from ultralytics import YOLO

# Export YOLO26n to NCNN, a lightweight runtime for ARM CPUs and mobile devices
model = YOLO("yolo26n.pt")
ncnn_path = model.export(format="ncnn")  # writes the yolo26n_ncnn_model folder

# Load the exported model and run it on this machine's CPU
ncnn_model = YOLO(ncnn_path)
results = ncnn_model("https://ultralytics.com/images/bus.jpg")
print(f"Inference: {results[0].speed['inference']:.1f} ms")

The export writes a yolo26n_ncnn_model folder holding model.ncnn.param for the network structure, model.ncnn.bin for the weights, and metadata.yaml for class names and input size. The printed time is the per-image inference latency on the machine that runs it, the number that decides whether a model fits an edge application's frame-rate budget.