FLOPs

FLOPs (floating-point operations) count the arithmetic a neural network performs to process one input, such as one image or one token, mostly the multiplications and additions inside its layers. The count measures a model's computational cost independently of the hardware it runs on, so it is the standard way to compare how heavy different architectures are. Vision models are usually quoted in GFLOPs (billions of operations) at a stated input size, and the total compute used to train a large model is quoted in FLOPs as well.

FLOPs vs. FLOPS vs. MACs#

Three similar terms are easy to confuse:

• FLOPs (lowercase s): the number of floating-point operations, a property of the model and input size. It measures how much work must be done.
• FLOPS (uppercase S): floating-point operations per second, a property of hardware such as a GPU. It measures how fast work can be done, and dividing a model's FLOPs by a device's peak FLOPS gives the shortest possible runtime.
• MACs: multiply-accumulate operations. One MAC is one multiplication plus one addition, so one MAC equals two FLOPs. Some papers and tools report MACs under the name FLOPs, which halves their numbers.

How FLOPs Are Counted#

Most of a network's FLOPs come from matrix multiplications and convolutions. A fully connected layer with n inputs and m outputs costs about 2 × n × m FLOPs per input. A convolution computes one dot product per output value, so its cost is 2 × C_in × k × k × C_out × H_out × W_out, where k is the kernel size and H_out × W_out the output resolution. Activations, normalization, and additions add comparatively little. The example below computes one convolution's cost by formula and checks it with PyTorch's built-in FLOP counter:

import torch
from torch.utils.flop_counter import FlopCounterMode

# A 3x3 convolution mapping 64 channels to 128 on an 80x80 feature map
conv = torch.nn.Conv2d(64, 128, kernel_size=3, padding=1, bias=False)
x = torch.randn(1, 64, 80, 80)

# Each output value needs 64*3*3 multiply-accumulates (MACs); 1 MAC = 2 FLOPs
macs = 128 * 80 * 80 * 64 * 3 * 3
print(f"By formula: {2 * macs / 1e9:.3f} GFLOPs ({macs / 1e9:.3f} GMACs)")

# PyTorch's FLOP counter traces the real forward pass
with FlopCounterMode(display=False) as counter:
    conv(x)
print(f"Counted:    {counter.get_total_flops() / 1e9:.3f} GFLOPs")

Both lines print 0.944 GFLOPs, and the formula line shows the same layer as 0.472 GMACs, half the FLOPs figure. For transformers, a forward pass costs about 2 FLOPs per parameter per token, and training costs about 6, a rule of thumb from the scaling laws research that estimates a training run's total compute as 6 × parameters × training tokens.

FLOPs and Real-World Speed#

FLOPs predict speed only roughly. Latency also depends on memory traffic, on how well each operation parallelizes, and on whether the hardware has optimized kernels for it. The ShuffleNet V2 paper showed that networks with similar FLOPs can run at clearly different speeds on the same device, and depthwise convolutions, cheap in FLOPs, are often limited by memory bandwidth instead of arithmetic. Optimizations also act on different factors. Model pruning and smaller input sizes remove operations, while model quantization leaves the operation count unchanged and makes each operation cheaper by computing in 8-bit integers. FLOPs therefore work best as a first filter before measuring inference latency on the target device, for example with Ultralytics benchmark mode:

FLOPs in Model Selection#

Model families are released in several sizes so teams can trade accuracy for compute, a choice the model size lesson in Ultralytics Academy walks through. Ultralytics YOLO26 ranges from 5.5 GFLOPs for YOLO26n to 194.4 GFLOPs for YOLO26x at a 640×640 input, and model.info() prints the GFLOPs of any loaded model. Battery-powered drones and phones use the smallest variants, while cloud servers can afford the largest. Because a convolutional network's cost scales with the number of pixels, input size changes FLOPs as much as model size does:

from ultralytics import YOLO
from ultralytics.utils.torch_utils import get_flops

# Load the smallest YOLO26 detection model and fuse its layers, as for inference
model = YOLO("yolo26n.pt")
model.fuse()

# Count GFLOPs for the same model at three input resolutions
for imgsz in [320, 640, 1280]:
    print(f"{imgsz}px: {get_flops(model.model, imgsz):.1f} GFLOPs")

The loop prints 1.3, 5.5, and 23.4 GFLOPs: each doubling of the input side multiplies the count by about four, so running YOLO26n at 1,280 pixels costs more than the 20.9 GFLOPs of YOLO26s, a model with four times the parameters, at 640.