Generative AI

Generative AI is the branch of artificial intelligence (AI) whose models learn the probability distribution of their training data and sample new data from it: text, images, audio, video, code, 3D scenes, or molecular structures. A discriminative model learns p(label | input), the probability of a label given an input, while a generative model learns p(input) itself or p(input | prompt), so it can produce inputs that never existed. The term entered everyday use after OpenAI released ChatGPT on November 30, 2022, but the methods are older: Diederik Kingma and Max Welling introduced the variational autoencoder in 2013, and Ian Goodfellow and colleagues introduced generative adversarial networks in 2014.

How Generative Models Work#

Four families of deep learning models produce most generative AI today, and every large language model (LLM) in the GPT mold belongs to the first:

• Autoregressive transformers: Generate a sequence one token at a time, each conditioned on everything written so far, for text, code, speech, and tokenized images.
• Diffusion models: Add noise to training samples and learn to remove it step by step, so generation starts from pure noise and denoises it into a sample. Stable Diffusion, released in August 2022, runs this process in a compressed latent space.
• Generative adversarial networks (GANs): Train a generator against a discriminator that tries to tell real samples from generated ones, which pushes the generator toward realistic output in a single forward pass.
• Variational autoencoders (VAEs): Encode data into a latent space with a known distribution and decode samples drawn from it. Latent diffusion models use a VAE to compress images before denoising.

The underlying idea fits in a few lines. A Gaussian mixture model learns the distribution of the 150 flowers in the Iris dataset and samples new ones, and a discriminative classifier trained on the same data labels them:

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.mixture import GaussianMixture

iris = load_iris()  # 150 flowers: sepal and petal length and width in cm

# Discriminative model: learns p(species | measurements), so it can only label flowers
classifier = LogisticRegression(max_iter=1000).fit(iris.data, iris.target)

# Generative model: learns p(measurements) itself, so it can sample new flowers
generator = GaussianMixture(n_components=3, random_state=0).fit(iris.data)
new_flowers, _ = generator.sample(3)

# The discriminative model labels the generated samples
for flower, species in zip(new_flowers.round(1), classifier.predict(new_flowers)):
    print(flower, iris.target_names[species])

The three sampled flowers, such as one with 4.7 cm sepals and 1.4 cm petals, were never measured, yet the classifier assigns them to versicolor, setosa, and virginica, because the mixture's three clusters line up with the three species. A text or image generator replaces the three Gaussians with a neural network of billions of parameters and the measurements with tokens or pixels.

Generative and Discriminative Models Together#

Image classification and object detection are discriminative tasks: YOLO26 maps an image to boxes and classes and cannot draw an image. The two kinds of model often work in sequence. Ultralytics Platform's class-prompted smart annotation uses generative vision-language models (VLMs), with Moondream 3.1 as the default, to draw boxes for a dataset's class names, and those reviewed boxes become labels for training a YOLO26 detector.

Generators also create synthetic data for rare events that are expensive or dangerous to photograph, such as unusual defects or road hazards, and synthetic data generation pipelines mix those images with real ones for training. The reverse direction is common too: detectors and classifiers score generated images for quality or flag unsafe content.

Applications#

• Text generation: Chat assistants, summarization, and translation run on LLMs, and coding assistants write and test software from a description.
• Text-to-image: Models such as Stable Diffusion create pictures from prompts, and the same models fill or replace masked regions of a photo.
• Text-to-video: Models such as OpenAI's Sora, previewed in February 2024, generate video clips from a description.
• Molecular structure: Google DeepMind and Isomorphic Labs' AlphaFold 3, published in May 2024, predicts the structures of protein complexes with a diffusion-based module that generates atom coordinates.

Risks and Limits#

Language generators hallucinate fluent but false statements, image and voice generators enable deepfakes, and training on web data raises copyright and consent questions. AI watermarking and C2PA content credentials mark generated media so it can be identified later. Generated content also feeds back into training data: Ilia Shumailov and colleagues showed in The Curse of Recursion, published in Nature in 2024, that models trained recursively on their own output lose the rare tails of the original distribution, a failure they named model collapse.