Deep Learning (DL)

Deep learning (DL) is a branch of machine learning (ML) that trains neural networks with many stacked layers to learn directly from raw data such as pixels, audio waveforms, and text. "Deep" refers to that depth: each layer transforms the output of the one before it, so the network builds its own hierarchy of features, from edges to textures to whole objects in an image, without engineers designing those features by hand. Deep learning powers modern computer vision, speech recognition, large language models (LLMs), and generative AI.

How Deep Learning Works#

A neural network is a chain of layers. Each layer multiplies its input by a matrix of learned weights, adds a bias, and passes the result through a nonlinear activation function such as ReLU. The nonlinearity is what makes depth useful: without it, any stack of layers collapses into a single linear transformation.

Training adjusts the weights so the network's outputs match the training labels. In each step, a batch of data flows forward through the network, a loss function scores the error, backpropagation computes how much each weight contributed to that error, and an optimizer such as stochastic gradient descent (SGD) or Adam nudges every weight in the direction that reduces it. Repeating this over many epochs turns random weights into useful features. The PyTorch example below trains a single linear layer and a three-layer network on points that are labeled by whether they fall inside a circle:

import torch
from torch import nn

torch.manual_seed(0)

# Points inside a circle are class 1: no straight line separates the classes
x = torch.rand(1000, 2) * 2 - 1
y = (x.pow(2).sum(dim=1) < 0.5).float().unsqueeze(1)

# A single linear layer versus stacked layers with nonlinear activations
models = {
    "linear": nn.Linear(2, 1),
    "deep": nn.Sequential(nn.Linear(2, 16), nn.ReLU(), nn.Linear(16, 16), nn.ReLU(), nn.Linear(16, 1)),
}
for name, model in models.items():
    optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
    for step in range(500):
        loss = nn.functional.binary_cross_entropy_with_logits(model(x), y)
        optimizer.zero_grad()
        loss.backward()  # backpropagation computes gradients for every layer
        optimizer.step()
    accuracy = ((model(x) > 0) == y).float().mean()
    print(f"{name}: {accuracy:.1%} accuracy")

The linear model reaches 60.8%, about the share of points outside the circle, because a straight boundary cannot enclose a region. The deep model reaches 99.9%: its hidden layers bend the decision boundary into a curve learned from the data. Ultralytics YOLO26 is trained with the same loop at much larger scale, and model.train() runs forward passes, loss computation, and backpropagation over labeled images on a GPU when one is available.

Deep Learning vs. Machine Learning#

Deep learning is a subset of machine learning, and the two differ mainly in where features come from. Classical algorithms such as decision trees, support vector machines, and gradient-boosted trees learn from features that people define, such as the size of a transaction or the count of a word. Deep networks learn the features themselves, which is why they dominate on unstructured data like images, audio, and text. That ability costs more: deep models typically need far more labeled data, train on GPUs or other accelerators, and expose less of their reasoning. On tabular data, gradient-boosted trees such as XGBoost remain strong competitors, and transfer learning narrows the data gap for deep models by starting from a network pretrained on a large dataset.

Key Architectures#

• Convolutional neural networks (CNNs): slide small learned filters across an image, so the same pattern is recognized anywhere in the frame. CNNs underpin real-time object detection and image segmentation.
• Recurrent neural networks (RNNs): process sequences one step at a time while carrying a hidden state. Variants such as LSTMs dominated speech and translation before transformers.
• Transformers: use self-attention to relate every element of a sequence to every other in parallel. Introduced in 2017, they power LLMs and Vision Transformers (ViTs).
• Diffusion models: learn to remove noise step by step, which lets them generate images, video, and audio from random noise.

A Brief History#

The ideas behind deep learning are decades old. Backpropagation was popularized by a 1986 paper from David Rumelhart, Geoffrey Hinton, and Ronald Williams, but networks stayed shallow until three ingredients arrived together: large labeled datasets such as ImageNet, GPUs fast enough to train on them, and techniques such as ReLU activations and dropout. In 2012, AlexNet cut the ImageNet top-5 error rate to 15.3%, against 26.2% for the next-best entry, and the field shifted to deep networks almost overnight. Hinton, Yann LeCun, and Yoshua Bengio received the 2018 Turing Award for this work, and the 2024 Nobel Prize in Physics went to Hinton and John Hopfield for discoveries that enable machine learning with artificial neural networks. Depth itself kept growing: residual connections let ResNet train networks of 152 layers, and an ensemble of them won the 2015 ImageNet classification challenge with a 3.57% top-5 error rate.