PyTorch

PyTorch is an open-source framework for building, training, and deploying neural networks in Python. It provides GPU-accelerated tensors, automatic differentiation, and a library of layers, optimizers, and loss functions, and it runs code eagerly, line by line, so models are written and debugged like ordinary Python programs. Created at Facebook AI Research, now Meta AI, PyTorch has been governed since 2022 by the vendor-neutral PyTorch Foundation under the Linux Foundation, and it is widely used for deep learning research and production models in computer vision and language.

Core Concepts#

• Tensors: multi-dimensional arrays, like NumPy arrays, that can live on a CPU or an accelerator. PyTorch supports NVIDIA GPUs through CUDA, AMD GPUs through ROCm, and Apple silicon through Metal (MPS), and moving data between them takes one .to(device) call.
• Autograd: as tensor operations run, PyTorch records them in a computation graph. Calling .backward() walks that graph in reverse and fills in the gradient of the output with respect to every tensor that requires one, which is the backpropagation step of training.
• Eager execution: the graph is rebuilt on every forward pass, so models can use Python loops, conditionals, and print statements, and errors point at the exact line that failed.
• Modules and optimizers: torch.nn provides layers such as convolutions, attention, and normalization, and torch.optim provides optimizers such as SGD and Adam that update weights from the computed gradients.
• torch.compile: added in PyTorch 2.0, it captures the eager program as a graph and generates fused kernels, often speeding up training and inference without changes to the model code.

The example below runs a tiny computation on a GPU if one is available and lets autograd compute its gradient:

import torch

# Use a GPU when PyTorch can see one, otherwise fall back to the CPU
device = "cuda" if torch.cuda.is_available() else "cpu"

# Tensors record the operations applied to them when requires_grad=True
w = torch.tensor([2.0, -1.0], device=device, requires_grad=True)
x = torch.tensor([3.0, 4.0], device=device)

# Ordinary Python runs eagerly: each line executes immediately
loss = (w * x).sum() ** 2

# Autograd walks the recorded graph backward to get d(loss)/dw
loss.backward()
print(f"device={device}, loss={loss.item()}, grad={w.grad.tolist()}")

On a machine without a GPU it prints device=cpu, loss=4.0, grad=[12.0, 16.0]. The loss is (2·3 − 1·4)² = 4, and autograd returns its derivative 2·(w·x)·x = [12, 16] without any hand-written calculus, the same mechanism that trains networks with billions of weights.

PyTorch vs. Other Tools#

• TensorFlow: Google's framework, which built static graphs before running them until TensorFlow 2 made eager execution the default. It remains common in production systems built around Keras and Google's deployment tools.
• JAX: Google's numerical library that transforms pure Python functions with composable grad, jit, and vmap operations compiled through XLA, popular for large-scale research on TPUs.
• OpenCV: a computer vision library for reading, transforming, and analyzing images with classical algorithms. Pipelines often use OpenCV to load and preprocess frames before a PyTorch model processes them.
• Hugging Face: a hub and set of libraries built largely on PyTorch, whose Transformers library supplies pretrained language and vision models as PyTorch modules.

From Research to Deployment#

A trained PyTorch model is saved as a checkpoint, usually a .pt or .pth file holding its weights. It can run in a Python service as is, but edge devices and optimized servers usually need it converted into a runtime built for their hardware:

Ultralytics YOLO26 is written in PyTorch: yolo26n.pt is a PyTorch checkpoint, training runs on PyTorch's autograd and optimizers, and model.export() converts the model into these deployment formats, including ONNX, TensorRT, OpenVINO, Core ML, and LiteRT.