LightGBM

LightGBM (Light Gradient Boosting Machine) is an open-source gradient boosting framework that trains ensembles of decision trees on tabular data, built for speed and low memory use on large datasets. Microsoft released it in 2016 and described its algorithms in the NeurIPS 2017 paper LightGBM: A Highly Efficient Gradient Boosting Decision Tree. In March 2026 the official repository moved to the lightgbm-org GitHub organization under the same maintainers. LightGBM handles classification, regression, and ranking from Python, R, C++, and the command line, and it appears in many winning solutions of machine learning competitions. On medium-sized tables of around 10,000 rows, tree ensembles like it still often outperform deep learning.

How LightGBM Works#

Gradient boosting builds trees one at a time. Each new tree is fit to the gradients of the loss function with respect to the current ensemble's predictions, which for squared error are simply the residuals, so it corrects the errors the earlier trees left. The model's prediction is the sum of all tree outputs, each scaled by a learning rate. The expensive step is finding the best split for each node, and LightGBM speeds it up with four techniques:

• Histogram-based splits: Before training, LightGBM buckets each continuous feature into at most 255 bins (max_bin). Split search scans bin boundaries instead of every sorted value, and after a split only the smaller child's histogram is built from data, since the larger child's equals the parent's minus the smaller one's.
• Leaf-wise growth: LightGBM always splits the leaf whose best split reduces the loss most, until the tree has num_leaves leaves (31 by default), instead of splitting every node at one depth before going deeper. For the same number of leaves this reaches a lower loss than level-wise growth, and it produces deep, lopsided trees that can overfit small datasets.
• Gradient-based One-Side Sampling (GOSS): Rows with large gradients are the ones the model still predicts badly. With data_sample_strategy="goss", LightGBM keeps the 20% of rows with the largest gradients, randomly samples 10% more from the rest, and up-weights that sample so the data distribution stays close to the original.
• Exclusive Feature Bundling (EFB): Sparse features that are rarely nonzero at the same time, such as one-hot columns, are merged into a single feature, which cuts the number of histograms to build.

Each tree grows by repeating the split search until its leaf budget runs out or no split helps, and boosting repeats that for every round:

LightGBM also splits integer-encoded categorical features directly when they are listed in categorical_feature, grouping categories at each split without one-hot encoding, and it handles missing values by default. Training runs on CPUs, on GPUs (device_type="gpu" or "cuda"), and across machines with its distributed learners.

Training and Tuning LightGBM#

The scikit-learn style LGBMClassifier, LGBMRegressor, and LGBMRanker classes cover most uses. Because leaf-wise trees can grow very deep, the main controls against overfitting are num_leaves, min_child_samples (20 rows per leaf by default), and max_depth, together with the learning rate and the number of trees, which hyperparameter tuning or early stopping on a validation set usually settles. The payoff of histograms and leaf-wise growth shows against scikit-learn's classic exact-split gradient boosting on a synthetic table of 20,000 rows and 40 features:

import time

import lightgbm as lgb
from sklearn.datasets import make_classification
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=20_000, n_features=40, n_informative=15, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)

# 200 trees each: exact split search vs. LightGBM's histograms and leaf-wise growth
for model in (GradientBoostingClassifier(n_estimators=200), lgb.LGBMClassifier(n_estimators=200, verbose=-1)):
    start = time.perf_counter()
    model.fit(X_train, y_train)
    seconds = time.perf_counter() - start
    print(f"{type(model).__name__}: {seconds:.1f} s, accuracy {model.score(X_test, y_test):.3f}")

On an Apple M5 Pro CPU, LightGBM grows its 200 trees of up to 31 leaves in under a second, while scikit-learn needs about 20 seconds for 200 exact-split trees of depth 3, and LightGBM's larger trees reach 0.956 test accuracy against 0.901. Trained models save to text or JSON, and onnxmltools converts them to ONNX for serving outside Python.

LightGBM vs. Related Methods#

• XGBoost: The gradient boosting library that popularized regularized, second-order boosting in 2014. It grows depth-limited trees by default (max_depth=6), offers leaf-wise growth with grow_policy="lossguide", and has used histogram split finding by default since version 2.0, so the two libraries now share most techniques.
• CatBoost: Yandex's boosting library, released in 2017, which builds symmetric trees and encodes categorical features with ordered target statistics to limit target leakage.
• Random forest: Trains deep trees independently on bootstrap samples and averages their predictions, which lowers variance.
• HistGradientBoosting: scikit-learn's HistGradientBoostingClassifier and HistGradientBoostingRegressor are implementations inspired by LightGBM that bin features into histograms the same way.
• Tabular foundation models: Pretrained transformers such as TabPFN predict on small tables in a single forward pass, with no per-dataset training run.

Real-World Applications#

• Fraud and credit risk: Banks score transactions and loan applications from account history, merchant, amount, and timing features, where LightGBM's fast inference fits per-transaction decisions.
• Demand forecasting: Retailers predict sales per product and store from lagged sales, prices, promotions, and calendar features. Foot traffic is a common input: Ultralytics YOLO26 object counting tracks people crossing a line in store video and reports in and out counts per class, which, summed per hour, become features for a LightGBM staffing forecast.
• Search and recommendation ranking: LGBMRanker trains with the LambdaRank objective by default, ordering documents or products within each query group to optimize normalized discounted cumulative gain (NDCG).