SigLIP

SigLIP (Sigmoid Loss for Language Image Pre-Training) is a method from Google researchers for training image and text encoders on image-caption pairs, and the name of the model family trained with it. Like CLIP, it learns a shared embedding space in which an image and a caption that describes it land close together. The difference is the loss function: SigLIP treats every image-text pair in a batch as its own yes-or-no question, scored with a sigmoid, instead of asking each image to pick its caption out of the whole batch with a softmax. Introduced in the 2023 SigLIP paper, this change lowers memory use, works better at small batch sizes, and made SigLIP encoders a common vision component of vision language models (VLMs).

How the Sigmoid Loss Works#

For a batch of N image-caption pairs, SigLIP computes the similarity of every image embedding with every text embedding, giving N×N scores. It multiplies each score by a learnable temperature t, adds a learnable bias b, and applies a binary log-sigmoid loss: the N true pairs on the diagonal are labeled as matches and the other N² − N pairs as non-matches. The temperature starts at 10 and the bias at −10, which starts training near the prior that almost every pair is a mismatch and avoids large early updates driven by the imbalance of one positive and N − 1 negatives per image.

The PyTorch code below computes this loss for four pairs, once with the true captions and once with the captions shifted by one position:

import torch
import torch.nn.functional as F

# Normalized embeddings for 4 image-caption pairs: row i of each tensor belongs together
torch.manual_seed(0)
images = F.normalize(torch.randn(4, 64), dim=-1)
texts = F.normalize(images + 0.05 * torch.randn(4, 64), dim=-1)

# Learnable temperature and bias, at the values SigLIP initializes them to
t, b = 10.0, -10.0

# Every image-text pair is its own binary problem: +1 for a match, -1 for the other 12 pairs
labels = 2 * torch.eye(4) - 1
for name, captions in [("matched", texts), ("shuffled", texts.roll(1, dims=0))]:
    logits = t * images @ captions.T + b
    loss = -F.logsigmoid(labels * logits).sum() / 4
    print(f"{name} captions: loss {loss:.2f}")

It prints matched captions: loss 1.21 and shuffled captions: loss 10.50. When the captions are shifted, every diagonal pair labeled as a match is a mismatch, and the loss rises almost ninefold.

Each pair's loss term depends only on that pair, so the full N×N matrix never has to exist on one device. Every accelerator scores its own images against its own texts, then passes its text embeddings to the next accelerator and repeats:

A softmax loss cannot be split this way, because each image's probability is normalized over every caption in the batch.

SigLIP vs. CLIP#

Both train two encoders with contrastive learning, and they differ in what the loss normalizes over:

• CLIP: A softmax cross-entropy over each row and each column of the batch's similarity matrix, computed after every device gathers the embeddings of all the others.
• SigLIP: An independent binary loss per pair. In the paper, SigLIP beat the softmax loss at small batch sizes, and gains from growing the batch flattened out, with a batch size of 32k sufficient. Combined with a frozen pretrained image encoder, it trained a model that reached 84.5% zero-shot ImageNet accuracy in two days on four TPUv4 chips.

The loss also shapes scores at inference: a SigLIP model gives each candidate label an independent match probability, so the probabilities over a set of labels need not sum to 1.

SigLIP 2#

SigLIP 2, released in February 2025, keeps the sigmoid loss and adds a captioning decoder, self-distillation, and masked prediction during training, along with online data curation. The encoders are multilingual and improve localization and dense features, which matter for detection and segmentation built on them. A NaFlex variant processes images at several resolutions while keeping their native aspect ratio, and the models come in ViT-B, L, So400m, and g sizes, where So400m is a 400-million-parameter shape-optimized ViT.

Real-World Applications#

• Vision encoders for VLMs: Google's PaliGemma uses a SigLIP So400m image encoder, and Gemma 3 uses a 400-million-parameter SigLIP encoder for its image inputs.
• Zero-shot classification: Labels written as text are scored against an image without training a classifier, in many languages with SigLIP 2.
• Semantic search: Image embeddings stored in a vector database are searched with text queries or with other images.
• Dataset exploration: Ultralytics Platform embeds every image of an analyzed dataset with an image-text encoder of this kind, so visually similar images sit together in its Clustering scatter plot, Find similar images returns the nearest look-alikes from other datasets, and a typed search such as "forklift near a doorway" also returns images that look like the description.