License YOLO
Back to Ultralytics Glossary

Huber Loss

Huber loss combines squared and absolute error to reduce outlier influence in regression, with a tunable delta threshold and Python implementation.

Huber loss is a regression loss function that penalizes small prediction errors quadratically and large errors linearly. It gives models a smooth objective near the correct answer while reducing the influence of unusually large errors. This makes it useful when most training examples follow a consistent pattern but some contain extreme deviations.

Peter J. Huber introduced the function in his 1964 paper Robust Estimation of a Location Parameter. It remains a foundational tool for robust estimation: fitting a model whose predictions are less sensitive to outliers.

How Huber Loss Works#

A residual is the predicted value minus its target. Huber loss uses a positive threshold called delta to determine how that residual contributes to the objective:

  • Small residuals: When the absolute residual is at most delta, the loss is half the squared residual.
  • Large residuals: Above delta, the loss is delta x (absolute residual - 0.5 x delta).

The two branches meet with the same value and slope. During backpropagation, the derivative with respect to the prediction equals the residual inside the quadratic region; outside it, its magnitude is capped at delta. Consequently, increasingly large residuals stop producing increasingly large prediction gradients. The linear tail limits how strongly an extreme error pulls the fit away from the remaining observations.

Choosing Delta and Comparing Losses#

Mean squared error (MSE) averages squared residuals and strongly emphasizes large errors. Mean absolute error (MAE), also called L1 loss when applied per element, uses absolute residuals throughout and has a sharp corner at zero. Huber loss combines quadratic behavior near zero with linear tails, providing a continuously differentiable objective for gradient descent.

A larger delta extends the squared-error region. A smaller delta moves more residuals into the outlier-resistant linear region and lowers its slope. Delta has the same units as the residual, so target normalization changes its meaning. Converting targets from meters to centimeters requires multiplying delta by 100 to preserve the same transition relative to the data.

Delta selection is a hyperparameter-tuning problem: candidate values are compared on validation data using the intended regression metric. Huber loss suits continuous-valued prediction; cross-entropy loss and hinge loss serve classification objectives.

Applications in Regression and Computer Vision#

In linear regression, extreme observations can pull a squared-error fit away from the main data trend. A HuberRegressor fit on a generated regression dataset with four added outliers follows the main trend more closely than a squared-error fit influenced by those observations.

For object detection, Ross Girshick's Fast R-CNN uses Smooth L1 loss, equivalent to unit-delta Huber loss, for bounding-box regression. Its linear tails reduce sensitivity to outlying coordinate errors, which otherwise produce large gradients during training.

Ultralytics YOLO supports custom training criteria through a model's init_criterion() method, providing an integration point for Huber-based custom regression objectives. Such customization changes how regression errors contribute to model updates.

Huber Loss in Python#

PyTorch provides torch.nn.HuberLoss with configurable delta and reduction. After installing torch with pip install torch, the following calculation exposes each residual's contribution:

import torch

predictions = torch.tensor([0.5, 2.0, 10.0])
targets = torch.zeros_like(predictions)

# Keep individual losses to expose the linear tail.
criterion = torch.nn.HuberLoss(delta=1.0, reduction="none")
huber_losses = criterion(predictions, targets)
squared_errors = (predictions - targets).square()

print("Huber losses:", huber_losses.tolist())
print("Squared errors:", squared_errors.tolist())

The output is [0.125, 1.5, 9.5] versus [0.25, 4.0, 100.0], showing the reduced penalty growth for large residuals. With reduction="mean", PyTorch averages all elementwise losses; "sum" adds them.

Pseudo-Huber loss approximates Huber's shape with a smooth transition and continuous higher-order derivatives. Smooth L1 uses a different scaling convention: in PyTorch, when beta equals delta, Huber loss equals Smooth L1 loss multiplied by delta; they are identical when both thresholds equal 1.

Explore solutions

Computer vision in sports

Computer vision in sports

Discover how Ultralytics YOLO powers sports vision with player and ball tracking, performance analytics, coaching insights, and athlete rehabilitation.
Learn more
Computer vision for aerial imagery

Computer vision for aerial imagery

Turn drone and aerial imagery into real-time insights for agriculture, aquaculture, conservation, and geospatial analysis with Ultralytics YOLO.
Learn more
Computer vision in aerospace

Computer vision in aerospace

Apply computer vision in aerospace with Ultralytics YOLO for aircraft inspection, runway FOD detection, airport ground operations, and composite defect detection.
Learn more
Computer vision in security

Computer vision in security

Protect every site with Ultralytics YOLO models. Vision AI powers perimeter monitoring, threat detection, license-plate recognition, and workplace safety.
Learn more
Computer vision in robotics

Computer vision in robotics

Power smarter machines with Ultralytics YOLO models. Vision AI in robotics drives autonomous navigation, perception, object tracking, and real-time control.
Learn more
Computer vision in logistics

Computer vision in logistics

Streamline logistics with Ultralytics YOLO models. Vision AI enables package inspection, sorting, vehicle tracking, and real-time warehouse safety monitoring.
Learn more
Computer vision in retail

Computer vision in retail

Reimagine retail with Ultralytics YOLO models. Vision AI powers inventory tracking, shelf monitoring, queue management, and smarter customer insights.
Learn more
Computer vision in healthcare

Computer vision in healthcare

Build healthcare solutions with Ultralytics YOLO models. Vision AI in healthcare powers faster medical imaging, smarter diagnostics, and patient monitoring.
Learn more
Computer vision in manufacturing

Computer vision in manufacturing

Optimize manufacturing with Ultralytics YOLO models. Vision AI drives quality control, defect detection, PPE compliance, and assembly line automation.
Learn more
Computer vision in automotive

Computer vision in automotive

Apply computer vision in automotive with Ultralytics YOLO models. Vision AI elevates road safety, driver assistance, and vehicle automation for smarter roads.
Learn more
Computer vision in agriculture

Computer vision in agriculture

Bring vision AI to smart agriculture with Ultralytics YOLO models. Power crop monitoring, livestock tracking, and precision farming for higher, smarter yields.
Learn more

Let's build the future of AI together!

Begin your journey with the future of machine learning