FP8

FP8 is an 8-bit floating-point number format used to train and run neural networks with half the memory and data movement of 16-bit formats. Like any floating-point type, each value has a sign bit, exponent bits that set its range, and mantissa bits that set its precision, but with only eight bits in total it trades a lot of both. Engineers from NVIDIA, Arm, and Intel defined the two standard variants, E4M3 and E5M2, in the September 2022 paper FP8 Formats for Deep Learning, and validated training of language models of up to 175 billion parameters with them. On GPUs with FP8 Tensor Cores, such as NVIDIA's Hopper H100 introduced in 2022, FP8 matrix multiplications run at twice the throughput of FP16 or BF16.

E4M3 and E5M2#

The two variants split the seven non-sign bits differently:

• E4M3: Four exponent bits and three mantissa bits. It gives up infinities and keeps a single NaN pattern to reach a maximum of 448, and it suits weights and activations in the forward pass.
• E5M2: Five exponent bits and two mantissa bits, following IEEE 754 conventions for special values. It reaches 57,344 with coarser steps, and suits gradients, whose magnitudes vary over a wider range.

Both formats flush small values to zero and clip large ones, so FP8 tensors usually carry a scale factor that shifts their values into the representable range. Casting gradient-sized values with and without one shows why:

import torch

torch.manual_seed(0)
grads = torch.randn(10_000) * 1e-4  # gradient-sized values, far below 1
e4m3, e5m2 = torch.float8_e4m3fn, torch.float8_e5m2
print(f"E4M3 max {torch.finfo(e4m3).max:g}, E5M2 max {torch.finfo(e5m2).max:g}")

# Per-tensor scaling maps the largest magnitude onto the E4M3 maximum before casting
scale = torch.finfo(e4m3).max / grads.abs().max()
results = {
    "E4M3 direct": grads.to(e4m3).float(),
    "E5M2 direct": grads.to(e5m2).float(),
    "E4M3 scaled": (grads * scale).to(e4m3).float() / scale,
}

for name, restored in results.items():
    zeros = (restored == 0).float().mean()
    error = ((restored - grads).abs() / grads.abs()).median()
    print(f"{name}: {zeros:.0%} rounded to zero, median relative error {error:.1%}")

A direct E4M3 cast rounds 100% of the values to zero, E5M2's wider range keeps most of them with a 6.4% median relative error and 7% lost, and scaling first lets E4M3 keep every value at a 2.2% median error. PyTorch also provides float8_e4m3fnuz and float8_e5m2fnuz, the variants AMD's MI300 accelerators use, which drop negative zero and shift the exponent bias, capping E4M3 at 240. An FP8 model is therefore not automatically portable between vendors.

Scaling Strategies#

One scale for a whole tensor fails when a few outliers set the maximum and push everything else toward zero, which is common in transformer activations. Recipes differ in how finely they scale:

• Per-tensor delayed scaling: NVIDIA's Transformer Engine picks each tensor's scale from the maximum values seen over recent training steps, so no extra pass over the data is needed.
• Block scaling: DeepSeek-V3 scaled activations per tile of 1 × 128 values and weights per 128 × 128 block, and its team reported the first validation of FP8 training on an extremely large model, 671 billion parameters.
• MXFP8: The Open Compute Project's microscaling format shares one power-of-two scale across every 32 values, and NVIDIA's Blackwell GPUs run it in hardware.

Matrix multiplications accumulate their results in higher precision, and layers that are sensitive or cheap, such as normalization, softmax, and the optimizer's master weights, stay in BF16 or FP32. FP8 is therefore one part of a mixed-precision recipe.

FP8 in Training and Inference#

For training, NVIDIA's Transformer Engine and TorchAO swap linear layers for FP8 versions while keeping master weights in higher precision. For inference, serving engines store model weights and the KV cache of large language models in FP8, which halves their memory compared with 16-bit formats and leaves room for more concurrent requests. TensorRT also builds FP8 engines for GPUs with FP8 Tensor Cores. The format needs hardware support to pay off: NVIDIA's A100 runs FP16, BF16, and INT8 on its Tensor Cores but has no FP8 units, while Ada Lovelace, Hopper, and Blackwell GPUs and AMD's MI300 series do.

FP8 sits between neighboring formats:

• FP16 or half precision: Twice the bits, with 10 mantissa bits and enough range for most inference without per-tensor scales.
• FP4: Half the bits again, relying on fine-grained block scales to stay accurate.
• INT8: The other 8-bit format, with 256 evenly spaced levels instead of FP8's exponent-based spacing, so it needs calibrated scales but runs on far more hardware, including most CPUs and NPUs.

Ultralytics YOLO26 takes the INT8 route to 8-bit inference: exporting with quantize=8 in export mode for formats such as TensorRT and OpenVINO runs post-training model quantization that calibrates scales on images from the data dataset, while a model fine-tuned with quantize=8 through quantization-aware training exports to TensorRT with the ranges it learned and needs no calibration.