TinyML

TinyML (tiny machine learning) is the practice of running machine learning models on microcontrollers and other ultra-low-power chips, typically with a few hundred kilobytes of memory, no operating system, and a power budget in milliwatts. Running inference on a chip right beside the sensor lets battery-powered devices listen, watch, or feel their surroundings continuously for months or years, sending only results such as "keyword heard" or "machine vibrating abnormally" instead of raw data. That keeps latency low and data private and saves the radio power that streaming would cost. The field's community organization, the EDGE AI FOUNDATION, was formerly called the tinyML Foundation.

How Models Fit on a Microcontroller#

A microcontroller stores a model's weights in flash memory alongside the program, and it holds the activations being computed in SRAM, which is usually the tighter limit. Model quantization to 8-bit integers cuts weights to a quarter of their FP32 size and replaces floating-point math with integer arithmetic that small cores run efficiently. Compact architectures built from depthwise-separable convolutions, model pruning, and lower input resolutions shrink the rest, and neural architecture search can design networks for a specific chip's memory. MCUNet combined such a search with a memory-aware runtime to reach over 70% ImageNet top-1 accuracy on an off-the-shelf microcontroller. The finished model is compiled into the firmware and executed by a small interpreter or compiled code:

The budget can be estimated before any hardware is involved. This keyword-spotting network, a depthwise-separable CNN of the kind used to recognize spoken commands from audio features, counts one byte per INT8 weight for flash and the largest pair of input and output activations for SRAM:

import torch
from torch import nn

# A depthwise-separable CNN for keyword spotting on 49x10 MFCC audio features
model = nn.Sequential(
    nn.Conv2d(1, 64, (10, 4), stride=2, padding=(5, 1)),
    nn.ReLU(),
    nn.Conv2d(64, 64, 3, padding=1, groups=64),
    nn.Conv2d(64, 64, 1),
    nn.ReLU(),
    nn.Conv2d(64, 64, 3, padding=1, groups=64),
    nn.Conv2d(64, 64, 1),
    nn.ReLU(),
    nn.AdaptiveAvgPool2d(1),
    nn.Flatten(),
    nn.Linear(64, 12),
)

# At INT8, flash stores one byte per weight and SRAM holds activations in flight
weights = sum(p.numel() for p in model.parameters())
x, peak = torch.randn(1, 1, 49, 10), 0
for layer in model:
    y = layer(x)
    peak = max(peak, x.numel() + y.numel())  # a layer's input and output coexist
    x = y
print(f"Flash: {weights / 1024:.1f} KB, peak SRAM: {peak / 1024:.1f} KB")

It prints 12.7 KB of flash and 15.6 KB of peak SRAM, a small fraction of the 1 MB of flash and 256 KB of RAM on the nRF52840 microcontroller of an Arduino Nano 33 BLE Sense, leaving room for the audio pipeline and the runtime itself.

Runtimes and Hardware#

• LiteRT for Microcontrollers: Google's interpreter, formerly TensorFlow Lite for Microcontrollers, needs no operating system, standard library, or dynamic memory allocation, and its core runtime fits in 16 KB on an Arm Cortex-M3.
• ExecuTorch: PyTorch's on-device runtime, whose Arm Ethos-U back end runs quantized models on Arm's Ethos-U NPUs, which sit beside Cortex-M cores in many low-power chips.
• Arm Cortex-M: The processor family inside most microcontrollers that run TinyML, from Cortex-M0+ parts with a few kilobytes of RAM up to the Cortex-M55 and Cortex-M85 with vector instructions for machine learning.
• Coral NPU: An open-source RISC-V accelerator design from Google, announced in October 2025 for always-on wearables, that targets about 512 billion operations per second while drawing a few milliwatts.

TinyML vs. Edge AI#

Edge AI covers any inference on the device where data is produced, including phones, single-board computers such as the Raspberry Pi, and embedded GPU modules such as NVIDIA Jetson, which have gigabytes of memory and draw watts. TinyML is its smallest tier, where models are measured in kilobytes. The boundary keeps moving as microcontrollers gain accelerators: ST's STM32N6 pairs a Cortex-M55 with a 600 GOPS neural processing unit and 4.2 MB of RAM. The smallest Ultralytics YOLO26 detector has 2.4 million parameters, about 2.4 MB at INT8, so its INT8 LiteRT export (format="litert", quantize=8) targets the tier above classic TinyML: phones, single-board computers, and accelerator-equipped boards.

Applications of TinyML#

• Keyword spotting: Always-on wake-word detectors in phones, earbuds, and smart speakers listen for a single phrase on a low-power core and wake the main processor only when they hear it.
• Predictive maintenance: Vibration and audio sensors bolted to motors and pumps learn normal operation and flag anomalies that precede failures, running for years on a battery.
• Wildlife and environmental monitoring: Solar or battery-powered acoustic sensors recognize animal calls or chainsaws in remote forests and send short alerts over long-range radio.
• Visual wake words: Low-resolution cameras on microcontrollers decide whether a person is in view and wake a larger system only when one is.

MLPerf Tiny, MLCommons' benchmark for this class of hardware, measures accuracy, latency, and energy on tasks drawn from these applications, such as keyword spotting, visual wake words, CIFAR-10 image classification, and anomaly detection on machine sounds.