Federated Learning

Federated learning is a way to train one machine learning model across many devices or organizations that keep their training data where it was collected. A coordinating server sends the current model to the participants, each one trains it on its own data, and only the resulting model updates travel back to be combined into a better shared model. Google researchers named the approach in their 2016 paper Communication-Efficient Learning of Deep Networks from Decentralized Data, which introduced the Federated Averaging (FedAvg) algorithm for training on data that stays on users' mobile devices. Bringing the model to the data lets phones, hospitals, and banks learn from information that privacy rules, regulations such as the EU's GDPR, or sheer size keep from being pooled in one data lake.

How Federated Learning Works#

Training runs in rounds. The server selects a group of available clients and sends them the global model weights. Each client runs a few epochs of gradient descent on its local data and returns its updated weights. FedAvg then averages the returned weights, giving each client a share proportional to its number of training examples, and the result becomes the next round's global model. Doing several local steps between communications cut the number of communication rounds by 10 to 100 times compared with synchronized stochastic gradient descent in the FedAvg experiments. This code simulates FedAvg for a linear model on three clients with different amounts of data:

import torch

torch.manual_seed(0)
true_w = torch.tensor([2.0, -3.0])

# Three clients with private datasets of different sizes; raw data never leaves a client
clients = []
for n in (50, 200, 500):
    X = torch.randn(n, 2)
    clients.append((X, X @ true_w + 0.1 * torch.randn(n)))

global_w = torch.zeros(2)
for r in range(1, 6):
    weighted = []
    for X, y in clients:
        w = global_w.clone()
        for _ in range(5):  # local gradient descent steps on the client's own data
            w -= 0.1 * 2 * X.T @ (X @ w - y) / len(y)
        weighted.append(len(y) * w)
    # FedAvg: average the client weights, weighted by each client's number of samples
    global_w = sum(weighted) / sum(len(y) for _, y in clients)
    print(f"round {r}: global weights {[round(v, 3) for v in global_w.tolist()]}")

It prints global weights of [1.354, -2.023] after round 1, [1.935, -2.898] after round 3, and [1.995, -2.99] after round 5, converging on the true values [2, -3] although the server only ever receives each client's weights.

Cross-Device and Cross-Silo Federated Learning#

Federated learning runs in two settings with different constraints. Cross-device training spans millions of phones or sensors, each holding a little data, often offline or on battery, so each round uses only a small fraction of them and must tolerate dropouts. Google's Gboard keyboard trains its next-word prediction models this way. Cross-silo training connects a few organizations, such as hospitals or banks, that each hold large datasets on reliable servers. In the Federated Tumor Segmentation study, 71 healthcare institutions on six continents trained a glioblastoma segmentation model on 25,256 MRI scans from 6,314 patients without sharing them, and the federated model delineated the surgically targetable tumor 33% better than a model trained on public data. Which setting fits depends on where the data sits:

The same loop works for vision models: each hospital, factory, or store can load the current global YOLO26 weights, fine-tune them on its own images with the standard Ultralytics YOLO train mode, and return only the updated weights for the server to average. Open-source frameworks such as Flower and NVIDIA FLARE handle client selection, communication, and aggregation for both settings.

Privacy and Security#

Keeping data local does not by itself make training private, because model updates carry information about the data that produced them. Gradient inversion attacks such as Deep Leakage from Gradients reconstructed training images and labels from shared gradients, and membership inference attacks test whether a specific record was used. Production systems add two defenses. Secure aggregation encrypts updates so the server learns only their sum, and differential privacy clips each update and adds calibrated noise so no single user's contribution can be recovered; Google reported that all of Gboard's next-word prediction neural network language models, which it trains with federated learning, now have differential privacy guarantees. The server must also guard against clients that send poisoned updates, using robust aggregation rules such as the coordinate-wise median that limit any one client's influence. Client data that differs widely between participants, such as one hospital's scanner versus another's, makes local models drift apart and slows convergence.

Federated Learning vs. Related Terms#

• Distributed training: Splits a centrally stored dataset across many GPUs in one data center to train faster. Federated learning cannot move the data, and its participants are slower, less reliable, and hold differently distributed data.
• Split learning: Divides the network itself between client and server and exchanges intermediate activations, while federated learning gives every client the full model and exchanges weights.
• Data privacy: The broader goal of protecting personal information. Federated learning is one technique for it and is usually combined with differential privacy and secure aggregation.
• Edge AI: Runs models on devices. Cross-device federated learning extends it by also training on those devices.