License YOLO
Back to Ultralytics Glossary

Self-Supervised Learning

Self-supervised learning (SSL) trains models on tasks whose labels come from the data itself, such as filling in masked parts. Covers methods and uses.

Self-supervised learning (SSL) trains a model on a task whose labels come from the data itself, such as predicting the next word of a sentence, filling in hidden image patches, or recognizing two crops of the same photo as a match, so it can learn from large amounts of unlabeled data. Supervised learning is limited by data labeling, which is slow and expensive, while raw images, video, audio, and text are plentiful. SSL uses them to pretrain a general-purpose encoder, and fine-tuning or a small trained head then adapts it to a downstream task such as image classification or object detection with far fewer labels. Every modern large language model (LLM) is pretrained this way, by next-token prediction over text as in GPT models, and Meta's DINOv3 scaled self-supervised image pretraining to a 7-billion-parameter model trained on 1.7 billion images.

How Self-Supervised Learning Works#

An SSL method defines a pretext task: it hides or transforms part of each input and trains the network to recover what was removed or to recognize what stayed the same. The pretext task itself is rarely useful. What matters is that solving it forces the network to learn the structure of the data, such as object shapes, textures, and how parts relate, and those features transfer to tasks with labels. After pretraining, the encoder is evaluated or used through linear probing on frozen features or by fine-tuning all of its weights:

Main Families of Methods#

  • Masked prediction: Hides part of the input and trains the model to reconstruct it. BERT predicts randomly masked words, and masked autoencoders hide 75% of an image's patches and rebuild their pixels. Predicting the next token of a sequence applies the same idea to text.
  • Contrastive learning: Creates two augmented views of each image and trains embeddings so that views of the same image match and views of different images do not. SimCLR and MoCo are the best-known examples.
  • Self-distillation: A student network learns to match the output of a teacher network, a slowly updated average of the student, on different crops of the same image, with no negative pairs. BYOL and DINO use this approach, and DINOv2 and DINOv3 scale it to curated datasets of over 100 million images.
  • Joint-embedding prediction: Predicts the embeddings of hidden regions from the visible ones, so the model learns abstract content without spending capacity on pixel detail. Meta's I-JEPA and V-JEPA apply it to images and video.

The choice shapes the features. Masked reconstruction keeps fine spatial detail and responds well to fine-tuning, while contrastive and self-distillation methods produce features that work well frozen, which is why DINO-family encoders are popular backbones for depth estimation and semantic segmentation.

Self-Supervised Learning in Computer Vision#

Vision transformers and SSL suit each other: masking whole patches maps directly onto a ViT's token inputs, and ViTs benefit from more training data than labeled datasets provide. Many foundation models start from self-supervised encoders. Meta's SAM 2, which runs in the Ultralytics package for promptable image and video segmentation, builds on a Hiera image encoder pretrained as a masked autoencoder before any segmentation training.

Applications#

  • Medical imaging: Hospitals hold far more scans than radiologists can annotate. Pretraining on unlabeled X-rays, CT, or MRI scans gives an encoder that later learns tasks such as tumor detection from a small set of expert labels.
  • Satellite imagery: Meta trained a DINOv3 variant on 493 million satellite images, and the World Resources Institute uses it to map tree canopy height. In a region of Kenya, it cut the average height error of earlier DINOv2-based maps from 4.1 to 1.2 meters.
  • Speech recognition: wav2vec 2.0 pretrained on 53,000 hours of unlabeled audio and, fine-tuned on only 10 minutes of transcribed speech, reached word error rates of 4.8% and 8.2% on the clean and other LibriSpeech test sets.
  • Unsupervised learning: Learning without human labels in general, including clustering and density estimation. SSL is usually counted as a form of unsupervised learning that is trained like supervised learning, with targets generated from the data.
  • Semi-supervised learning: Trains on a small labeled set and a large unlabeled set at the same time. SSL pretrains on unlabeled data alone, and the two combine well: SimCLRv2 fine-tuned a large self-supervised model on 1% of ImageNet's labels and then distilled it using the unlabeled images.
  • Transfer learning: Reuses a pretrained model for a new task. SSL is one way to produce that pretrained model without labels.
  • Data augmentation: Random crops, color changes, and flips that expand a labeled dataset. In contrastive SSL, the same transformations define which views count as the same image, so they decide what the encoder learns to ignore.

Explore solutions

Computer vision in sports

Computer vision in sports

Discover how Ultralytics YOLO powers sports vision with player and ball tracking, performance analytics, coaching insights, and athlete rehabilitation.
Learn more
Computer vision for aerial imagery

Computer vision for aerial imagery

Turn drone and aerial imagery into real-time insights for agriculture, aquaculture, conservation, and geospatial analysis with Ultralytics YOLO.
Learn more
Computer vision in aerospace

Computer vision in aerospace

Apply computer vision in aerospace with Ultralytics YOLO for aircraft inspection, runway FOD detection, airport ground operations, and composite defect detection.
Learn more
Computer vision in security

Computer vision in security

Protect every site with Ultralytics YOLO models. Vision AI powers perimeter monitoring, threat detection, license-plate recognition, and workplace safety.
Learn more
Computer vision in robotics

Computer vision in robotics

Power smarter machines with Ultralytics YOLO models. Vision AI in robotics drives autonomous navigation, perception, object tracking, and real-time control.
Learn more
Computer vision in logistics

Computer vision in logistics

Streamline logistics with Ultralytics YOLO models. Vision AI enables package inspection, sorting, vehicle tracking, and real-time warehouse safety monitoring.
Learn more
Computer vision in retail

Computer vision in retail

Reimagine retail with Ultralytics YOLO models. Vision AI powers inventory tracking, shelf monitoring, queue management, and smarter customer insights.
Learn more
Computer vision in healthcare

Computer vision in healthcare

Build healthcare solutions with Ultralytics YOLO models. Vision AI in healthcare powers faster medical imaging, smarter diagnostics, and patient monitoring.
Learn more
Computer vision in manufacturing

Computer vision in manufacturing

Optimize manufacturing with Ultralytics YOLO models. Vision AI drives quality control, defect detection, PPE compliance, and assembly line automation.
Learn more
Computer vision in automotive

Computer vision in automotive

Apply computer vision in automotive with Ultralytics YOLO models. Vision AI elevates road safety, driver assistance, and vehicle automation for smarter roads.
Learn more
Computer vision in agriculture

Computer vision in agriculture

Bring vision AI to smart agriculture with Ultralytics YOLO models. Power crop monitoring, livestock tracking, and precision farming for higher, smarter yields.
Learn more

Let's build the future of AI together!

Begin your journey with the future of machine learning