License YOLO
Back to Ultralytics Glossary

Generative Flow Networks (GFlowNets)

Generative flow networks (GFlowNets) build objects step by step and sample each one in proportion to its reward, yielding diverse, high-scoring candidates.

A generative flow network (GFlowNet) is a machine learning model that learns to build discrete objects, such as molecules, graphs, or sequences, through a series of actions, and to sample each finished object with probability proportional to a given reward. A reinforcement learning agent trained to maximize the same reward tends to settle on the single best object it has found, while a GFlowNet returns many different good objects in proportion to how good they are. That diversity matters when a reward is only a cheap proxy for the real goal, as when a model proposes drug candidates for laboratory testing. Unlike most generative AI models, a GFlowNet needs no training data, only the reward function. Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio of Mila introduced GFlowNets at NeurIPS 2021 in Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation.

Flows Through a Network of States#

Generation is a sequential decision process, like a Markov decision process (MDP) without randomness in the transitions. It starts from an empty state, and each action adds one building block, such as an atom, a molecular fragment, a graph edge, or a token, until the policy chooses to stop. Different action orders can reach the same object, so the states form a directed acyclic graph instead of a tree:

The name comes from picturing probability as water flowing from the start state through this graph and draining out of each finished object at a rate equal to its reward. If the inflow equals the outflow at every state, a forward policy that follows each edge with probability proportional to its flow produces every object with probability equal to its reward divided by the total flow Z, the sum of all rewards. The original paper turned this flow-consistency condition into a training loss function, analogous to the Bellman equation of temporal-difference learning.

Training With Trajectory Balance#

Nikolay Malkin and colleagues' trajectory balance objective (2022) checks a whole trajectory at once instead of every state: the learned total flow Z times the product of forward-policy probabilities along the trajectory must equal the reward times the product of backward-policy probabilities, which describe how the finished object could be taken apart. The loss is the squared difference of the logarithms of the two sides. When every object has exactly one construction path, as with strings built left to right, the backward policy is 1 and drops out, as in this PyTorch example:

import torch

torch.manual_seed(0)
reward = torch.tensor([1.0, 1, 1, 8, 1, 4, 1, 16])  # reward of each 3-bit string 000 ... 111
logits = torch.zeros(7, 2, requires_grad=True)  # forward policy: P(next bit) at each of the 7 prefixes
log_z = torch.zeros(1, requires_grad=True)  # learned log of the total flow Z
optimizer = torch.optim.Adam([logits, log_z], lr=0.05)

for step in range(2000):
    node, log_pf = 0, 0.0
    for depth in range(3):  # build the string one bit at a time, walking down a binary tree
        policy = torch.distributions.Categorical(logits=logits[node])
        bit = policy.sample()
        log_pf = log_pf + policy.log_prob(bit)
        node = 2 * node + 1 + bit.item()
    # Trajectory balance loss; each string has one path in a tree, so the backward policy is 1
    loss = (log_z + log_pf - reward[node - 7].log()) ** 2
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

# Probability of sampling each string = product of the policy's choices along its path
probs = logits.softmax(dim=1).detach()
sampled = [probs[0, x >> 2] * probs[1 + (x >> 2), (x >> 1) & 1] * probs[3 + (x >> 1), x & 1] for x in range(8)]
print("P(x):    ", [round(p.item(), 2) for p in sampled])
print("R(x)/sum:", [round(r, 2) for r in (reward / reward.sum()).tolist()])
print(f"Learned Z = {log_z.exp().item():.1f}, sum of rewards = {reward.sum():.0f}")

After 2,000 sampled strings, the policy's probabilities match the reward shares to two decimals, from 0.03 for each reward-1 string to 0.48 for 111, whose reward of 16 is about half of the total, and the learned Z equals the sum of rewards, 33. Real GFlowNets replace the lookup table with a neural network, often a graph neural network for molecules, and explore with random actions or replayed high-reward objects, because the policy only learns about objects it samples. A reward maximizer on the same task would put almost all its probability on 111. Hyperparameter tuning in Ultralytics YOLO26 applies reward-weighted sampling by hand: model.tune() picks the parent of each new candidate from its 9 best results with probability weighted by fitness, then mutates it, so the search keeps exploring around several good configurations.

Applications#

  • Drug discovery: The original paper built molecules from fragments, scored by a proxy model of binding to the soluble epoxide hydrolase (sEH) protein, and found more distinct high-reward molecules than reinforcement learning and Markov chain Monte Carlo baselines. Moksh Jain and colleagues applied GFlowNets to designing antimicrobial peptides in 2022.
  • Causal discovery: DAG-GFlowNet (Tristan Deleu and colleagues, 2022) approximates the posterior distribution over the graph structures of Bayesian networks, which keeps several plausible causal explanations in view.
  • Language model reasoning: Edward Hu and colleagues fine-tuned language models with GFlowNet objectives in 2024 to sample reasoning chains and text infills in proportion to their posterior probability.
  • Reward-maximizing reinforcement learning: Learns a policy over the same kind of decision process but maximizes expected reward, which concentrates probability on the best object found.
  • Markov chain Monte Carlo: Also samples in proportion to a reward, but each sample needs a long chain of proposals; a trained GFlowNet produces one in a single pass, which is the "non-iterative" in the original title.
  • Evolutionary algorithms: Search with hand-designed mutation and fitness-weighted selection instead of a learned construction policy.
  • Normalizing flows: Share the word "flow" but transform continuous distributions through invertible layers.

Explore solutions

Computer vision in sports

Computer vision in sports

Discover how Ultralytics YOLO powers sports vision with player and ball tracking, performance analytics, coaching insights, and athlete rehabilitation.
Learn more
Computer vision for aerial imagery

Computer vision for aerial imagery

Turn drone and aerial imagery into real-time insights for agriculture, aquaculture, conservation, and geospatial analysis with Ultralytics YOLO.
Learn more
Computer vision in aerospace

Computer vision in aerospace

Apply computer vision in aerospace with Ultralytics YOLO for aircraft inspection, runway FOD detection, airport ground operations, and composite defect detection.
Learn more
Computer vision in security

Computer vision in security

Protect every site with Ultralytics YOLO models. Vision AI powers perimeter monitoring, threat detection, license-plate recognition, and workplace safety.
Learn more
Computer vision in robotics

Computer vision in robotics

Power smarter machines with Ultralytics YOLO models. Vision AI in robotics drives autonomous navigation, perception, object tracking, and real-time control.
Learn more
Computer vision in logistics

Computer vision in logistics

Streamline logistics with Ultralytics YOLO models. Vision AI enables package inspection, sorting, vehicle tracking, and real-time warehouse safety monitoring.
Learn more
Computer vision in retail

Computer vision in retail

Reimagine retail with Ultralytics YOLO models. Vision AI powers inventory tracking, shelf monitoring, queue management, and smarter customer insights.
Learn more
Computer vision in healthcare

Computer vision in healthcare

Build healthcare solutions with Ultralytics YOLO models. Vision AI in healthcare powers faster medical imaging, smarter diagnostics, and patient monitoring.
Learn more
Computer vision in manufacturing

Computer vision in manufacturing

Optimize manufacturing with Ultralytics YOLO models. Vision AI drives quality control, defect detection, PPE compliance, and assembly line automation.
Learn more
Computer vision in automotive

Computer vision in automotive

Apply computer vision in automotive with Ultralytics YOLO models. Vision AI elevates road safety, driver assistance, and vehicle automation for smarter roads.
Learn more
Computer vision in agriculture

Computer vision in agriculture

Bring vision AI to smart agriculture with Ultralytics YOLO models. Power crop monitoring, livestock tracking, and precision farming for higher, smarter yields.
Learn more

Let's build the future of AI together!

Begin your journey with the future of machine learning