Model Weights

Model weights are the learnable numbers inside a neural network that transform its inputs into predictions. Each weight scales the signal passing between two units, or forms one value of a convolution filter, and together the weights encode everything the model has learned from its training data. The term is often used loosely for all of a model's learned parameters, including biases and normalization values, as in "pretrained weights" or "open-weight models". Weight counts span orders of magnitude: ResNet-50 has about 25.6 million parameters, while large language models have billions.

How Training Sets the Weights#

A new network starts with small random weights drawn from schemes such as He initialization, which scale the random values to each layer's size so signals neither vanish nor explode as they pass through many layers. At this point the model's predictions are noise. Training then repeats a cycle for each batch of data: a forward pass produces predictions, a loss function scores their error, backpropagation computes how much each weight contributed to that error, and an optimizer such as stochastic gradient descent (SGD) or the Adam optimizer nudges every weight to reduce it. After millions of these small updates, the weights settle into values that produce accurate predictions, and they stay fixed during inference.

Weights vs. Related Terms#

• Biases: In a layer that computes y = Wx + b, the weights W multiply the inputs and the bias b adds an offset, which lets a unit's activation function shift its threshold and produce a nonzero output when every input is zero. Both are learned during training.
• Hyperparameters: Settings such as the learning rate and batch size that are chosen before training and control how the weights are learned. Gradient descent does not update them.
• Activations: The intermediate outputs a network computes for a particular input. Weights stay the same for every input, while activations are recomputed for each one.
• Weights & Biases: An experiment-tracking platform whose name borrows the two parameter types.

Saving and Loading Weights#

Trained weights are saved to files so they can be reloaded for inference, shared, or used as the starting point for transfer learning. In PyTorch, a model's weights live in its state_dict and are usually saved to .pt files. These files use Python's pickle format, which can execute code when loaded, so since PyTorch 2.6 torch.load defaults to weights_only=True. The safetensors format stores raw tensors only, which rules out code execution and allows fast memory-mapped loading, and it is the standard format on Hugging Face. ONNX files package the weights together with the computation graph so other runtimes can execute the model.

Ultralytics YOLO26 .pt checkpoints store an exponential moving average (EMA) of the weights, which smooths out step-to-step noise, together with the training arguments and per-epoch results, and every training run writes best.pt and last.pt to its weights folder. Loading the same architecture with and without trained weights shows where the model's knowledge lives:

from ultralytics import YOLO

# Same YOLO26n architecture with two different sets of weights
trained = YOLO("yolo26n.pt")  # weights learned on COCO
untrained = YOLO("yolo26n.yaml")  # randomly initialized weights

for name, model in [("trained", trained), ("untrained", untrained)]:
    results = model("https://ultralytics.com/images/bus.jpg")
    print(f"{name}: {len(results[0].boxes)} objects detected")

The trained weights find the bus and the people in the image, while the randomly initialized copy of the identical network detects nothing.

Compressing Weights for Deployment#

Weight precision sets a model's file size and memory footprint, which matters most for edge AI devices. Model quantization stores weights at lower precision, and moving from 32-bit floating point to 8-bit integers makes them four times smaller and speeds up inference on hardware with integer support. Pruning removes weights that contribute little to the output, and knowledge distillation trains a smaller model to mimic a larger one. For scale, a 7-billion-parameter language model needs about 28 GB for its weights at 32 bits, 14 GB at 16 bits, and 3.5 GB at 4 bits.