Generative Flow Networks (GFlowNets)
Generative flow networks (GFlowNets) build objects step by step and sample each one in proportion to its reward, yielding diverse, high-scoring candidates.
A generative flow network (GFlowNet) is a machine learning model that learns to build discrete objects, such as molecules, graphs, or sequences, through a series of actions, and to sample each finished object with probability proportional to a given reward. A reinforcement learning agent trained to maximize the same reward tends to settle on the single best object it has found, while a GFlowNet returns many different good objects in proportion to how good they are. That diversity matters when a reward is only a cheap proxy for the real goal, as when a model proposes drug candidates for laboratory testing. Unlike most generative AI models, a GFlowNet needs no training data, only the reward function. Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio of Mila introduced GFlowNets at NeurIPS 2021 in Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation.
Flows Through a Network of States#
Generation is a sequential decision process, like a Markov decision process (MDP) without randomness in the transitions. It starts from an empty state, and each action adds one building block, such as an atom, a molecular fragment, a graph edge, or a token, until the policy chooses to stop. Different action orders can reach the same object, so the states form a directed acyclic graph instead of a tree:
The name comes from picturing probability as water flowing from the start state through this graph and draining out of each finished object at a rate equal to its reward. If the inflow equals the outflow at every state, a forward policy that follows each edge with probability proportional to its flow produces every object with probability equal to its reward divided by the total flow Z, the sum of all rewards. The original paper turned this flow-consistency condition into a training loss function, analogous to the Bellman equation of temporal-difference learning.
Training With Trajectory Balance#
Nikolay Malkin and colleagues' trajectory balance objective (2022) checks a whole trajectory at once instead of every state: the learned total flow Z times the product of forward-policy probabilities along the trajectory must equal the reward times the product of backward-policy probabilities, which describe how the finished object could be taken apart. The loss is the squared difference of the logarithms of the two sides. When every object has exactly one construction path, as with strings built left to right, the backward policy is 1 and drops out, as in this PyTorch example:
import torch
torch.manual_seed(0)
reward = torch.tensor([1.0, 1, 1, 8, 1, 4, 1, 16]) # reward of each 3-bit string 000 ... 111
logits = torch.zeros(7, 2, requires_grad=True) # forward policy: P(next bit) at each of the 7 prefixes
log_z = torch.zeros(1, requires_grad=True) # learned log of the total flow Z
optimizer = torch.optim.Adam([logits, log_z], lr=0.05)
for step in range(2000):
node, log_pf = 0, 0.0
for depth in range(3): # build the string one bit at a time, walking down a binary tree
policy = torch.distributions.Categorical(logits=logits[node])
bit = policy.sample()
log_pf = log_pf + policy.log_prob(bit)
node = 2 * node + 1 + bit.item()
# Trajectory balance loss; each string has one path in a tree, so the backward policy is 1
loss = (log_z + log_pf - reward[node - 7].log()) ** 2
optimizer.zero_grad()
loss.backward()
optimizer.step()
# Probability of sampling each string = product of the policy's choices along its path
probs = logits.softmax(dim=1).detach()
sampled = [probs[0, x >> 2] * probs[1 + (x >> 2), (x >> 1) & 1] * probs[3 + (x >> 1), x & 1] for x in range(8)]
print("P(x): ", [round(p.item(), 2) for p in sampled])
print("R(x)/sum:", [round(r, 2) for r in (reward / reward.sum()).tolist()])
print(f"Learned Z = {log_z.exp().item():.1f}, sum of rewards = {reward.sum():.0f}")After 2,000 sampled strings, the policy's probabilities match the reward shares to two decimals, from 0.03 for each reward-1 string to 0.48 for 111, whose reward of 16 is about half of the total, and the learned Z equals the sum of rewards, 33. Real GFlowNets replace the lookup table with a neural network, often a graph neural network for molecules, and explore with random actions or replayed high-reward objects, because the policy only learns about objects it samples. A reward maximizer on the same task would put almost all its probability on 111. Hyperparameter tuning in Ultralytics YOLO26 applies reward-weighted sampling by hand: model.tune() picks the parent of each new candidate from its 9 best results with probability weighted by fitness, then mutates it, so the search keeps exploring around several good configurations.
Applications#
- Drug discovery: The original paper built molecules from fragments, scored by a proxy model of binding to the soluble epoxide hydrolase (sEH) protein, and found more distinct high-reward molecules than reinforcement learning and Markov chain Monte Carlo baselines. Moksh Jain and colleagues applied GFlowNets to designing antimicrobial peptides in 2022.
- Causal discovery: DAG-GFlowNet (Tristan Deleu and colleagues, 2022) approximates the posterior distribution over the graph structures of Bayesian networks, which keeps several plausible causal explanations in view.
- Language model reasoning: Edward Hu and colleagues fine-tuned language models with GFlowNet objectives in 2024 to sample reasoning chains and text infills in proportion to their posterior probability.
GFlowNets vs. Related Terms#
- Reward-maximizing reinforcement learning: Learns a policy over the same kind of decision process but maximizes expected reward, which concentrates probability on the best object found.
- Markov chain Monte Carlo: Also samples in proportion to a reward, but each sample needs a long chain of proposals; a trained GFlowNet produces one in a single pass, which is the "non-iterative" in the original title.
- Evolutionary algorithms: Search with hand-designed mutation and fitness-weighted selection instead of a learned construction policy.
- Normalizing flows: Share the word "flow" but transform continuous distributions through invertible layers.










