Containerization

Containerization is a way of packaging software in which an application and everything it needs to run, including its runtime, libraries, system packages, and configuration, are bundled into an image that runs as an isolated process called a container. Each container sees its own filesystem, process tree, and network interfaces, while all containers on a host share that host's operating system kernel, which makes them much lighter than virtual machines. In machine learning (ML), containerization removes conflicts between projects that need different versions of Python, CUDA, or PyTorch, and lets a model trained on a workstation run unchanged on a server, a cluster, or an edge computing device.

How Containers Work#

Linux containers are built mainly on two kernel features. Namespaces control what a process can see, giving it its own process IDs, network stack, mount points, and users, and cgroups (control groups) control what it can use by capping CPU, memory, and I/O. The idea grew out of chroot in 1979 and FreeBSD jails in 2000, cgroups entered the Linux kernel in 2008, and Docker made containers mainstream in 2013. The Open Container Initiative (OCI), founded in 2015, standardized the image and runtime formats, so an image built with one tool runs under another.

A container runtime does the work. High-level runtimes such as containerd and CRI-O pull images and manage container lifecycles, and a low-level OCI runtime such as runc creates the namespaces and cgroups and starts the process. Because Linux containers need a Linux kernel, macOS and Windows run them inside a small Linux virtual machine.

Images, Layers, and Registries#

An image is a stack of read-only layers plus a configuration that names the command to run. At startup, the runtime merges the layers into one filesystem, usually with OverlayFS, and adds a thin writable layer for that container, so many containers share one copy of the image on disk. Registries such as Docker Hub, GitHub Container Registry, and each cloud provider's registry store images under tags such as latest and under content digests, SHA-256 hashes that always identify the same bytes. Builds reuse unchanged layers, which is why Dockerfiles install dependencies before copying frequently edited code:

Containerization in ML Workflows#

In MLOps pipelines, containers carry a project from experiment to production. Framework images such as those in the NVIDIA NGC Catalog pin CUDA, cuDNN, and framework versions, so every node of a distributed training job runs the same software, and the host supplies only the GPU driver, which the NVIDIA Container Toolkit exposes to the container. Continuous integration tests run inside the same image that later ships, inference servers for model serving run as images that Kubernetes replicates and scales with traffic, and edge AI devices such as NVIDIA Jetson boards update a model by pulling a new image tag.

The On Premise integration of Ultralytics Platform for Enterprise workspaces installs Docker if needed and runs one worker container from the official Ultralytics image that matches the host's ARM64, CPU, or NVIDIA GPU hardware. The worker mounts the dataset folder read-only, so source images stay on that computer while training metrics and the best checkpoint of each YOLO26 run go to Platform.

Containers vs. Virtual Machines#

A virtual machine runs a complete guest operating system with its own kernel on a hypervisor, so every VM carries an OS image of several gigabytes and boots like a separate computer. A container is an ordinary process on the shared host kernel, so it starts as fast as its application and adds only its own files. Sharing the kernel also means a kernel vulnerability can expose every container on a host, so the two are often combined: public clouds run customer containers inside VMs, and sandboxed runtimes add a second boundary for untrusted code, such as the code that AI agents write and run in agent sandboxes. Google's gVisor intercepts a container's system calls with its own user-space kernel. Kata Containers runs each container inside a lightweight virtual machine, and AWS built its Firecracker microVMs to isolate AWS Lambda functions the same way.