Machine Learning Operations (MLOps)

Machine learning operations (MLOps) is the set of practices and tools for building, deploying, and maintaining machine learning (ML) models in production reliably and repeatably. It extends DevOps, which automates how software is tested and released, to systems whose behavior depends on data as well as code: a model can degrade without a single line changing because the world it observes has changed. The model itself is a small part of such a system. Google's MLOps architecture guide notes that only a small fraction of a real-world ML system is ML code, with data collection and verification, serving infrastructure, and monitoring making up the rest, and the NeurIPS paper Hidden Technical Debt in Machine Learning Systems described the long-term maintenance costs that this surrounding system accumulates.

Core Practices#

MLOps connects data engineering, model development, and operations through a few practices, each applied to the training data as well as the code:

• Data versioning: Every dataset snapshot gets an immutable version, so any model can be traced to the exact data that produced it.
• Experiment tracking: Each training run logs its code version, data version, hyperparameters, metrics, and artifacts, so results can be compared and reproduced.
• Continuous integration and delivery: Automated pipelines test code, validate incoming data, retrain models, and block any candidate that fails a quality gate.
• Model registry: Approved model versions are stored with their metadata, lineage, and stage, such as staging or production, so deployments and rollbacks name exact versions.
• Model serving: Models are packaged, often in containers, and exposed through APIs or deployed to edge devices.
• Model monitoring: Production inputs, predictions, latency, and accuracy are tracked to catch drift and decide when to retrain.

The Google guide describes maturity in three levels. At level 0, data scientists train and hand over models manually. Level 1 automates the training pipeline itself, adding continuous training (CT), the automatic retraining and serving of models when new data arrives or data drift is detected. Level 2 also automates building, testing, and deploying the pipeline code with CI/CD, so new pipeline versions reach production as safely as new models. Together, these practices form a loop:

MLOps in Practice#

• Manufacturing inspection: When a factory adds a product line, new defect images are labeled and versioned, a pipeline retrains the detection model, and the approved version rolls out to edge cameras on every line without downtime.
• Retail shelf monitoring: Packaging redesigns and store lighting shift what cameras see. Monitoring flags falling confidence scores, low-confidence images go to annotation, and the retrained model ships to stores once it passes evaluation.

A quality gate automates the promotion decision. The following script trains a candidate object detection model, scores it and the current production model on the same validation data, and exports the candidate to ONNX only if its mAP_50-95 is higher:

from ultralytics import YOLO

# Train a candidate; the run saves its weights, args.yaml, and results.csv under project/name
candidate = YOLO("yolo26n.pt")
candidate.train(data="coco8.yaml", epochs=10, project="mlops", name="candidate")

# Quality gate: score the candidate and the production model on the same validation data
candidate_map = candidate.val(data="coco8.yaml").box.map
production_map = YOLO("yolo26n.pt").val(data="coco8.yaml").box.map

# Promote only an improvement, exported in a format the serving stack runs
if candidate_map > production_map:
    candidate.export(format="onnx")

In a production pipeline, a scheduler or CI job runs a script like this whenever new labeled data lands, and the exported model is registered with its data version and metrics. Ultralytics Platform covers this lifecycle for vision models in one workspace, with immutable dataset versions, cloud training with live metrics, export to formats such as TensorRT, dedicated inference endpoints, and monitoring of deployed models.

MLOps vs. Related Terms#

• DevOps: Automates building, testing, and releasing software, where a code change triggers the pipeline. MLOps adds data and models as versioned assets, so new data or a drop in production accuracy can trigger it too.
• LLMOps: The branch of MLOps for large language model applications, centered on prompts, retrieval, evaluation of generated text, and token costs.
• Automated machine learning (AutoML): Automates building a model, such as choosing algorithms and hyperparameters. MLOps runs the lifecycle around it, from data versioning to monitoring, and often calls AutoML as one pipeline step.
• MLflow and Weights & Biases: Tools that cover parts of the lifecycle, such as tracking and model registries. MLOps is the practice that combines such tools with pipelines, serving, and monitoring.