Text-to-Image

Text-to-image generation is the task of creating a new image from a natural-language description, called a prompt. A text-to-image model learns how words relate to visual content from hundreds of millions to billions of image-caption pairs, then turns a prompt such as "a red bicycle leaning against a brick wall at sunset" into a picture that never existed. It is a core application of generative AI that joins natural language processing (NLP) and computer vision. Most current systems, including Stable Diffusion, FLUX, and Google's Imagen, are diffusion or flow models, while others, such as the image generation OpenAI built into GPT-4o in 2025, produce images autoregressively as sequences of tokens.

How Text-to-Image Models Work#

A text encoder first turns the prompt into embeddings. Early Stable Diffusion versions used OpenAI's CLIP text encoder, and Google's Imagen showed that enlarging its frozen language encoder, up to the T5-XXL model, improved both image quality and image-text alignment much more than enlarging the image generator. The generator then starts from random noise and removes it over 20 to 50 steps, usually in the compressed space of a latent diffusion model, with cross-attention or joint attention layers letting every part of the image attend to the prompt. Classifier-free guidance compares the model's prediction with and without the prompt and amplifies the difference, so a higher guidance scale follows the prompt more closely at the cost of variety. A decoder finally turns the finished latent into pixels.

Training data shapes what a model can draw. Stable Diffusion was trained on subsets of LAION-5B, 5.85 billion image-text pairs collected from the web, and OpenAI's DALL-E 3 improved prompt following by training on detailed synthetic captions instead of the short alt text that accompanies most web images. Web-scale data also carries problems: LAION withdrew LAION-5B in December 2023 after a Stanford Internet Observatory report found links to child sexual abuse material in it, and released the cleaned Re-LAION-5B in August 2024.

Model Families#

• Diffusion and flow models: Denoise an image or latent over many steps. This family includes DALL-E 2 and 3, Imagen, Stable Diffusion, and FLUX, and its newer members replace the U-Net denoiser with a diffusion transformer trained with rectified flow.
• Autoregressive models: Turn images into discrete tokens and predict them one at a time with a transformer, the approach of the original DALL-E in 2021 and Google's Parti. OpenAI describes GPT-4o's image generation, launched in March 2025, as an autoregressive model natively embedded within ChatGPT, unlike the diffusion-based DALL-E.
• Generative adversarial networks: Generate an image in one forward pass. Text-conditioned GANs such as AttnGAN came before diffusion models and are faster to sample but follow complex prompts less reliably.

Evaluating Generated Images#

Fréchet Inception Distance (FID) compares the statistics of generated and real images to measure quality and diversity, and CLIP score measures how well an image matches its prompt in a shared embedding space. Neither checks specifics such as how many objects appear or where, so GenEval runs an object detection model on generated images to verify object co-occurrence, position, and count, and adds other vision models to check color. Public arenas rank commercial and open models from many thousands of pairwise human votes, which capture aesthetics and prompt adherence together.

Text-to-Image vs. Related Terms#

• Image captioning: The reverse direction, generating a text description of an existing image.
• Text-to-video: Generates a sequence of frames that must stay consistent in time, which multiplies the compute and data a text-to-image model needs.
• Image inpainting and image-to-image: Start from an existing picture and change part or all of it according to a prompt, using the same models with an extra image input.
• Vision-language models: Read images and text and answer in text. Natively multimodal models such as GPT-4o and Gemini have added image generation to image understanding.

Applications and Risks#

Designers, game studios, and marketing teams use text-to-image models to explore concepts and produce product imagery, and vision teams use them to create synthetic data for rare classes, defects, and conditions that are hard to photograph. On Ultralytics Platform, the Generate Similar Images action describes new scenes inspired by a dataset image and generates one image per description, with optional instructions for lighting, viewpoint, background, or objects, and the images a user keeps join the dataset unlabeled, ready for annotation. Synthetic images help only when they match what the model is missing:

The same realism creates risks. Models reproduce the biases of their training data, and they can produce convincing deepfakes. Providers respond with AI watermarking such as Google's SynthID and with C2PA Content Credentials, signed metadata that records which tool created or edited an image.