Split Learning

Split learning is a collaborative distributed training method that divides a neural network at a chosen cut layer between a data owner, the client, and a server. The client runs the layers before the cut on its private data and sends only the resulting activations, sometimes called smashed data; the server runs the remaining layers, computes the loss, and sends back the gradient at the cut so the client can update its layers. Raw training data never leaves the client, and neither side holds the whole model. Otkrist Gupta and Ramesh Raskar of the MIT Media Lab introduced it in October 2018 in Distributed Learning of Deep Neural Network over Multiple Agents, and the group's December 2018 follow-up, Split Learning for Health, proposed configurations that let hospitals holding different kinds of patient data, such as scans for medical image analysis, train models together without sharing raw records.

How Split Learning Works#

Each training step crosses the cut twice: activations go from client to server in the forward pass, and the gradient of the loss with respect to those activations comes back during backpropagation. Because the chain rule only needs that one gradient to continue, the client's updates are identical to training the unsplit model. A single-process simulation in PyTorch shows the exchange:

import torch
from torch import nn

torch.manual_seed(0)
client = nn.Sequential(nn.Linear(784, 256), nn.ReLU())  # layers before the cut, beside the private data
server = nn.Sequential(nn.Linear(256, 10))  # layers after the cut
images, labels = torch.randn(32, 784), torch.randint(0, 10, (32,))

# Client: forward pass to the cut layer, then send only the activations
activations = client(images)
received = activations.detach().requires_grad_()

# Server: finish the forward pass and backpropagate to the cut, then send that gradient back
nn.functional.cross_entropy(server(received), labels).backward()

# Client: continue backpropagation from the returned gradient
activations.backward(received.grad)
split_grad = client[0].weight.grad.clone()

# The same step on the unsplit model gives the client identical gradients
client.zero_grad()
nn.functional.cross_entropy(server(client(images)), labels).backward()
print("Same client gradients as unsplit training:", torch.equal(split_grad, client[0].weight.grad))

It prints Same client gradients as unsplit training: True. Each side then steps its own optimizer. With several clients, the original protocol trains them in turn, each picking up the latest client-side weights from the previous one.

The arrangement changes with what must stay private. The vanilla setup above sends labels to the server, a U-shaped setup keeps the last layers and labels on the client too, and a vertical setup serves organizations that hold different features about the same records:

Split Learning vs. Federated Learning#

Federated learning also keeps data on its owners' machines, but each client trains a complete model copy and sends weight updates for averaging. Split learning sends activations and gradients instead, so clients need far less compute and never hold the full model, while communication grows with the amount of data rather than the model size, and clients train one after another instead of in parallel. SplitFed learning, proposed by Chandra Thapa and colleagues in 2020, combines the two by running client-side layers in parallel and averaging them federated-style.

Vertical split learning also serves finance, where a bank and a partner holding different features about the same customers train one model, as in SecretFlow's bank marketing tutorial, without exchanging private data. Pipeline parallelism cuts a network by depth in the same way, but across GPUs of one trusted cluster to fit a large model in memory.

Privacy Risks#

Activations and gradients still carry information about the data. Early-layer activations can be inverted back into recognizable images, and when the server holds the labels, the gradients it returns can reveal them to the client. Dario Pasquini, Giuseppe Ateniese, and Massimo Bernaschi showed at ACM CCS 2021 that a malicious server can hijack training through the gradients it returns, steering the client's layers into a form that lets the server reconstruct private inputs. Defenses add noise with differential privacy, penalize how much the activations reveal about the input, or detect unusual gradients, each at some cost in accuracy. Deployments pair split learning with data security controls such as authentication and encrypted transport, since the protocol itself only governs what crosses the cut.

Choosing the Cut Layer#

The cut sets the balance between client compute, traffic, and exposure: an early cut needs the least client compute but sends large activations that still resemble the input, while a later cut sends more abstract feature maps but asks more of client hardware, which matters for edge AI devices. Skip connections complicate the choice, because every tensor that crosses the cut must be sent. Cutting Ultralytics YOLO26 after its backbone means sending three feature maps, since its model configuration feeds the outputs of layers 4, 6, and 10 into the neck: for YOLO26n at imgsz=640 they hold 1,126,400 values, nearly as many as the 1,228,800 in the input image.