Super Resolution

Super resolution (SR) is a family of computer vision techniques that reconstruct a higher-resolution image or video from a lower-resolution input. Classic upscaling such as bicubic interpolation only spreads the existing pixels over a larger grid, so edges turn soft and fine textures blur. A learned SR model predicts the missing high-frequency detail, such as sharp edges, text strokes, and fabric or foliage textures, from patterns it learned on pairs of low- and high-resolution images. Single-image super resolution (SISR) works from one picture, and video super resolution also draws on neighboring frames, which each capture slightly different information about the same scene.

How Super Resolution Works#

Super resolution is an ill-posed problem: many different high-resolution images shrink to the same low-resolution one, so the model has to choose the most plausible answer using what it learned about natural images. Training pairs are made by taking high-resolution images and degrading them with blur, downsampling by a factor such as 2× or 4×, noise, and compression. That degradation model decides which real inputs the network can handle; Real-ESRGAN trains on randomly chained degradations so it can restore real photos and video as well as cleanly downsampled images.

Architectures have followed the rest of deep learning. SRCNN showed in 2014 that a three-layer convolutional neural network (CNN) beats classical methods. ESPCN runs its convolutions at low resolution and upscales only at the end with a sub-pixel convolution, which rearranges channels into a larger image, making it fast enough for real-time 1080p video. EDSR stacked many residual blocks, SwinIR built its model from shifted-window vision transformer blocks, and diffusion models generate detail through iterative denoising.

The loss function decides what "better" means. Pixel losses such as L1 or mean squared error maximize fidelity to the original but average over the possible answers, which looks smooth. SRGAN added a perceptual loss on deep features and an adversarial loss from a generative adversarial network (GAN), producing sharp, realistic textures that may not match the true scene pixel for pixel, a risk in medical and forensic imaging, where an invented detail can be mistaken for evidence.

Measuring Super Resolution#

Fidelity is measured with peak signal-to-noise ratio (PSNR), a log-scale measure of mean squared error in decibels, and with the structural similarity index (SSIM). Perceptual metrics such as LPIPS compare deep features and track human judgments of realism more closely. The comparison below shrinks an image 4× and rebuilds it with bicubic interpolation and with a pretrained ESPCN model from OpenCV:

import urllib.request

import cv2  # cv2.dnn_superres requires: pip install opencv-contrib-python

# Download a pretrained 4x ESPCN model and a test image cropped to a multiple of 4
urllib.request.urlretrieve("https://github.com/fannymonori/TF-ESPCN/raw/master/export/ESPCN_x4.pb", "ESPCN_x4.pb")
urllib.request.urlretrieve("https://ultralytics.com/images/bus.jpg", "bus.jpg")
original = cv2.imread("bus.jpg")[:1080, :808]

# Simulate a low-resolution capture by shrinking the image 4x
low_res = cv2.resize(original, (202, 270), interpolation=cv2.INTER_CUBIC)

# Upscale 4x with bicubic interpolation and with the learned ESPCN model
bicubic = cv2.resize(low_res, (808, 1080), interpolation=cv2.INTER_CUBIC)
sr = cv2.dnn_superres.DnnSuperResImpl_create()
sr.readModel("ESPCN_x4.pb")
sr.setModel("espcn", 4)
espcn = sr.upsample(low_res)

# Higher PSNR means the reconstruction is closer to the original image
print(f"Bicubic: {cv2.PSNR(original, bicubic):.2f} dB, ESPCN: {cv2.PSNR(original, espcn):.2f} dB")

It prints Bicubic: 19.51 dB, ESPCN: 20.52 dB. The 1 dB gain corresponds to about 21% lower mean squared error, from a model file of roughly 100 KB.

Super Resolution vs. Related Terms#

• Image restoration: The broader field of recovering a clean image from a degraded one, including denoising, deblurring, and compression artifact removal. Super resolution is the restoration task that also increases the pixel count, and modern models often handle several degradations at once.
• Image inpainting: Fills in missing or masked regions, while super resolution adds detail everywhere in an image that is complete but coarse.
• Frame interpolation: Raises temporal resolution by synthesizing frames between existing ones, the time-axis counterpart of spatial super resolution.
• Generative AI: Text-to-image models create images from prompts. Super resolution is conditioned on an input image and must stay faithful to its content, and many image generators use a diffusion upscaler as their final stage.

Real-World Applications#

• Gaming: NVIDIA DLSS renders frames at a lower resolution and upscales them with a neural network, raising frame rates at a similar visual quality.
• Video streaming and archives: Broadcasters and film restorers upscale standard-definition and archival footage to HD and 4K displays.
• Satellite image analysis: Analysts sharpen lower-cost imagery to map buildings, roads, and vehicles.
• Medical image analysis: Research systems reconstruct high-resolution MRI from faster, lower-resolution acquisitions, shortening the time a patient spends in the scanner.

Super resolution also serves as a preprocessing step for detecting small objects in low-resolution sources. Ultralytics YOLO26 resizes every input to its imgsz, 640 pixels by default, with bilinear interpolation, so a 320×240 camera frame is enlarged 2× before detection; a 2× super resolution model in place of that interpolation hands the detector reconstructed edges on small, distant objects such as faces, license plates, and vehicles in aerial views.