Object Detection

Object detection is the computer vision task of finding every instance of a set of known classes in an image or video frame and reporting where each one is. For every object it returns a bounding box, a class label, and a confidence score, so a street photo becomes a list of labeled boxes, one per car, person, or traffic light. That list is the input to counting, measuring, tracking, and robot control.

What a Detector Returns#

Each detection has three parts. The box is stored either as corner coordinates (x1, y1, x2, y2) or as a center point with a width and height, in pixels or normalized to the image size. The class is an index into the model's label set, such as the 80 categories of the COCO dataset. The score, between 0 and 1, ranks how likely the box is to contain that class, and applications keep only detections above a confidence threshold, trading missed objects against false alarms.

from ultralytics import YOLO

# Load a YOLO26 nano detector pretrained on the 80 COCO classes
model = YOLO("yolo26n.pt")

# Run detection on one image
results = model("https://ultralytics.com/images/bus.jpg")
result = results[0]

# Each detection carries a class, a confidence score, and a box in pixels
for box in result.boxes:
    name = result.names[int(box.cls)]
    corners = box.xyxy[0].int().tolist()
    print(f"{name} {box.conf.item():.2f} {corners}")

Each printed line is one detected object: its class name, its confidence, and the pixel coordinates of its top-left and bottom-right corners.

How Object Detectors Work#

Modern detectors are neural networks with three stages. A backbone, usually a convolutional network or a vision transformer, turns the image into feature maps. A neck such as a feature pyramid network merges those maps across scales so that small and large objects are both represented. A detection head then predicts class scores and box coordinates at thousands of positions. Neighboring positions fire on the same object, so the raw output holds many overlapping candidates, and most pipelines drop low-scoring boxes and run non-maximum suppression (NMS) to keep one box per object. End-to-end detectors such as DETR are trained to emit exactly one prediction per object, so they skip that step.

Detector designs differ in how they find candidates. Two-stage detectors such as Faster R-CNN first propose regions that might hold an object, then classify and refine each one. One-stage detectors such as YOLO, introduced in 2015, predict all boxes and classes in a single pass, which made real-time detection practical. Earlier one-stage models regressed offsets from predefined anchor boxes, and anchor-free detectors predict the distances from a grid point to the four box edges directly. YOLO26 is a one-stage, anchor-free detector that applies NMS by default, and passing nms=False switches it to a one-to-one head that outputs final boxes with no NMS step.

Training and Evaluation#

Detectors learn from images annotated with a box and class for every object. During training, a label-assignment step decides which predictions are responsible for each labeled object, and the loss combines a classification term with a box-regression term, usually built on intersection over union (IoU), the overlap between a predicted box and a labeled box. Evaluation uses IoU again: a detection counts as correct when its class matches and its IoU with a labeled box reaches a threshold. Mean average precision (mAP) then summarizes precision and recall across every confidence level and class. COCO reports mAP₅₀, scored at an IoU of 0.5, and uses mAP_50-95, averaged over ten IoU thresholds from 0.50 to 0.95, as its primary metric. The VLM Leaderboard uses mAP₅₀ to compare vision language models that detect objects from class names alone. Ultralytics Platform uses such models to auto-annotate detection datasets before YOLO26 trains on the labels.

Real-World Applications#

• Autonomous driving: Driver-assistance and self-driving systems detect vehicles, pedestrians, cyclists, and traffic signs in every camera frame, and tracking and motion planning act on those boxes.
• Retail: Shelf cameras detect products and empty facings to flag out-of-stock items, and self-checkout systems detect the items placed at the register.
• Manufacturing inspection: Line cameras detect scratches, dents, and missing components so that faulty parts are rejected as they pass.

Object Detection vs. Related Tasks#

• Image classification assigns one label to a whole image and gives no location.
• Instance segmentation outlines each detected object with a pixel mask that follows its exact shape.
• Oriented bounding boxes add a rotation angle to each box so it fits angled objects, such as ships and vehicles in aerial images.
• Object tracking links detections of the same object across video frames and gives each one a persistent ID.
• 3D object detection predicts a box in 3D space, described by its center, its length, width, and height in meters, and its heading angle.