Docker

Docker is an open-source platform for building, sharing, and running applications in containers: isolated processes that carry their own filesystem with code, libraries, and system packages while sharing the host's operating system kernel. dotCloud, the company that later renamed itself Docker, Inc., open-sourced it in March 2013, and its command-line tool, layered image format, and public registry made Linux containers an everyday developer tool. In machine learning, a Docker image pins the exact Python version, CUDA libraries, PyTorch build, and model weights, so a model behaves the same on a laptop, a GPU server, a continuous integration runner, and an edge device.

How Docker Works#

A Dockerfile lists the steps that build an image: a base image to start from, commands that install packages, files to copy in, and the command to run at startup. Each RUN, COPY, or ADD instruction adds a read-only layer, and the next build reuses cached layers until an instruction or one of its input files changes, so dependencies installed early stay cached while application code rebuilds in seconds. An image is that stack of layers plus metadata. A container is a running instance of an image with a thin writable layer on top, which is deleted with the container unless the data goes to a mounted volume.

The docker command talks to Docker Engine, a background daemon that hands each container to containerd and the low-level runc runtime. These use Linux namespaces to isolate a container's processes, network, and filesystem, and cgroups to limit its CPU and memory. Images are shared through registries such as Docker Hub, GitHub Container Registry, NVIDIA NGC, and each cloud provider's own registry. They follow the Open Container Initiative image format, so Podman, containerd, and Kubernetes run them unchanged.

Ultralytics publishes official Docker images for each hardware target, including CUDA GPUs, CPUs, ARM64 boards, NVIDIA Jetson devices, and AMD GPUs, each also tagged with the Ultralytics release number for reproducible builds. The lightweight latest-python image used below already contains Python, PyTorch, the ultralytics package, and YOLO26 nano weights, so a project's Dockerfile only adds its own code:

# Official Ultralytics CPU image with Python, PyTorch, ultralytics, and yolo26n.pt
FROM ultralytics/ultralytics:latest-python

# Install extra dependencies in their own layer, cached until requirements.txt changes
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy the application last, so code edits rebuild only this layer
COPY app.py .

# Command every container runs when it starts
CMD ["python", "app.py"]

docker build -t yolo-app . turns this file and the files beside it into an image, and docker run --rm yolo-app starts a container from it. Pushing the image to a registry lets any other machine pull and run the identical environment, for example as a model serving API or a nightly inference job.

Docker for Machine Learning#

Containers reach GPUs through the NVIDIA Container Toolkit. The NVIDIA driver stays on the host, and the image carries only CUDA user-space libraries, which must be supported by that driver. On Linux, Docker 28.2 and later enable CDI by default, so --device nvidia.com/gpu=all requests GPUs as CDI devices and keeps GPU access in long-running containers, which the older --gpus all flag can lose when the host reloads systemd.

Two other settings matter for training. PyTorch data loader workers pass batches to the training process through shared memory in /dev/shm, which Docker limits to 64 MB by default, so training containers run with --ipc=host or a larger --shm-size. Datasets and outputs belong in mounted volumes, since a training run that writes checkpoints inside the container loses them when the container is removed. Pinning a release tag or image digest instead of latest keeps a distributed training job or a model deployment on the same image across every machine, and both inference servers such as NVIDIA Triton Inference Server and managed services such as Vertex AI run models as container images.

Docker vs. Virtual Machines and Kubernetes#

• Virtual machines: A hypervisor emulates hardware, and every VM boots its own guest kernel, which isolates workloads more strongly and allows any guest operating system at the cost of a full OS image per VM and slower startup. Because containers share the host's Linux kernel, Docker Desktop on macOS and Windows runs them inside a lightweight Linux VM.
• Docker Compose: Defines several containers that run together on one host, such as an inference API, a cache, and a metrics collector.
• Kubernetes: Schedules containers across a cluster, restarts failed ones, and scales replicas with traffic. Kubernetes 1.24 removed dockershim, its built-in adapter for Docker Engine, so clusters run containers through containerd or CRI-O, and Docker-built images still run there because they are standard OCI images.