Dynamic Resolution Scaling
Dynamic resolution scaling changes the resolution of each frame's work on the fly to hold a frame-time budget, used in games and real-time vision AI.
Dynamic resolution scaling (DRS) is a technique that adjusts the resolution at which a system processes each frame, frame by frame, to keep the time per frame within a fixed budget. When a scene gets expensive, the renderer or model works on fewer pixels; when there is headroom, it raises the resolution again, and the output is usually upscaled back to a constant display size. Video games popularized it to hold a steady frame rate on consoles and PCs, and the same idea applies to real-time inference, where a computer vision model can shrink its input to stay within a latency budget on busy or thermally throttled hardware.
How Dynamic Resolution Scaling Works#
DRS is a feedback loop. Each frame, the system measures how long the last frames took on the GPU, compares that with the budget, and picks a resolution scale for the next frame within fixed bounds:
The gap between "over" and "well under" keeps the resolution from flickering between two levels. In Unreal Engine, for example, the default budget is 33.3 ms per frame, or 30 frames per second, and the screen percentage moves between 50% and 100% of the output resolution along each axis, so the cheapest setting renders a quarter of the pixels.
The upscaling step decides how much quality the lower resolution costs. Simple spatial scaling blurs the image, so modern games pair DRS with learned or temporal upscalers such as NVIDIA DLSS, AMD FSR, Intel XeSS, and Sony's PlayStation Spectral Super Resolution, which combine previous frames and, in most current versions, deep learning to reconstruct detail. That reconstruction is a form of super-resolution, and it lets a game render well below its output resolution with little visible loss.
Dynamic Resolution in Vision AI#
For a convolutional neural network, compute and inference latency grow roughly with the number of input pixels, so halving the input side to 320 pixels cuts the FLOPs of a 640-pixel model to about a quarter. A controller can therefore trade resolution for speed whenever an edge AI device such as an NVIDIA Jetson heats up and throttles its clock, a server takes on more camera streams, or a drone needs spare compute for navigation. The cost is accuracy on small and distant objects, which cover too few pixels at low resolution, so object detection systems keep a floor on the input size.
Ultralytics YOLO26 supports this directly in predict mode: a PyTorch model accepts a different imgsz on every call without reloading, resizing each frame's longer side to imgsz and padding the shorter side only to the next multiple of its 32-pixel stride, while models exported to static-shape formats keep the size they were exported with unless dynamic=True was set. A simple controller steps through allowed sizes based on measured inference time:
import cv2
from ultralytics import YOLO
model = YOLO("yolo26n.pt")
cap = cv2.VideoCapture("https://github.com/ultralytics/assets/releases/download/v0.0.0/decelera_portrait_min.mov")
sizes = [320, 416, 512, 640] # allowed input sizes, multiples of the 32-pixel stride
level, budget_ms = len(sizes) - 1, 15.0 # start at full size with a 15 ms inference budget
frames_at = dict.fromkeys(sizes, 0)
while True:
ok, frame = cap.read()
if not ok:
break
result = model(frame, imgsz=sizes[level], verbose=False)[0]
frames_at[sizes[level]] += 1
# Step down when over budget, and back up only when well under it to avoid flickering
if result.speed["inference"] > budget_ms and level > 0:
level -= 1
elif result.speed["inference"] < 0.6 * budget_ms and level < len(sizes) - 1:
level += 1
print("Frames processed at each input size:", frames_at)The printout counts the frames processed at each size, and slower hardware spends more of the video at the smaller sizes.
Related Terms#
- Super-resolution: Reconstructs a higher-resolution image from a lower one, and is often the second half of a DRS pipeline.
- Adaptive bitrate streaming: Switches video quality to match network bandwidth rather than compute time.
- Variable rate shading: Lowers shading detail in selected regions of a frame instead of scaling the whole frame.
- Multi-scale training: Trains a model on varying input sizes so it stays accurate when inference resolution changes.
- Native dynamic resolution in vision-language models: Encodes each image at its own resolution into a variable number of tokens to preserve detail, as in Alibaba's Qwen2-VL, which calls its version Naive Dynamic Resolution.










