Keypoints

Keypoints are specific, named points on an object, such as a person's left wrist, the center of a car wheel, or the corners of a box, stored as image coordinates. A pose estimation model predicts a fixed set of them for every object it finds, each with a confidence score, and edges between them form a skeleton that captures posture and shape. Keypoints mark an object's parts, a finer description than a bounding box around the whole object or a segmentation mask of its outline. Landmarks is a synonym, used mostly for faces.

Defining and Labeling Keypoints#

Each dataset fixes its keypoint set and order. COCO defines 17 keypoints for the human body: the nose, eyes, ears, shoulders, elbows, wrists, hips, knees, and ankles. Every label stores an x and y coordinate plus a visibility flag: 0 for not labeled, 1 for labeled but hidden, and 2 for labeled and visible. Other sets cover other subjects, such as the 21 points of the hand keypoints dataset, the 68-point face scheme of 300-W, 24 points per dog in Dog-Pose, or the corners of a manufactured part. Because the order carries meaning, data augmentation that flips an image horizontally must also swap every left keypoint with its right counterpart, or a flipped left wrist would keep the label of the other arm.

How Models Predict Keypoints#

Deep learning models predict keypoints either as heatmaps, one map per keypoint whose peak marks its location, or by regressing the coordinates directly from the image features. Each prediction comes with a confidence that drops for keypoints hidden behind another person, cut off by the frame edge, or blurred by motion, so applications can ignore low-confidence points instead of trusting a guess. Predictions are scored with object keypoint similarity (OKS), which tolerates larger errors on keypoints that people label less consistently, such as hips, than on precise ones, such as eyes.

from ultralytics import YOLO

# Load a YOLO26 pose model that predicts the 17 COCO body keypoints per person
model = YOLO("yolo26n-pose.pt")
results = model("https://ultralytics.com/images/bus.jpg")
keypoints = results[0].keypoints

# COCO order: index 0 is the nose, 9 and 10 are the left and right wrists
for index, name in [(0, "nose"), (9, "left wrist"), (10, "right wrist")]:
    x, y = keypoints.xy[0, index].tolist()
    print(f"{name}: ({x:.0f}, {y:.0f}), confidence {keypoints.conf[0, index]:.2f}")

Each line gives one keypoint of the first detected person as a pixel position and a confidence, and keypoints.xyn holds the same positions normalized to the image size.

Semantic Keypoints vs. Interest Points#

Pose keypoints are semantic: a model learns what a left wrist looks like. Interest points are found by a detector instead of defined in advance. SIFT (1999) finds blobs that stay stable across scale and rotation, ORB (2011) finds corners fast enough for real-time mapping, and SuperPoint (2018) learns to find repeatable points. A descriptor computed around each point lets matchers such as LightGlue link the same point across images for image registration, 3D reconstruction, and visual SLAM, and point tracking follows such points through video.

Measuring with Keypoints#

Keypoints turn an image into geometry: two keypoints give a distance, three give a joint angle, and a sequence over frames gives speed. The Ultralytics workouts monitoring solution counts exercise repetitions this way, by default measuring the angle at the right elbow from keypoints 6, 8, and 10 (right shoulder, elbow, and wrist) of a YOLO26 pose model for every tracked person:

• Sports analysis: Joint angles and limb speeds from keypoints describe a golf swing, a sprinter's stride, or a pitcher's throwing motion frame by frame.
• Driver monitoring: Eye and mouth landmarks measure how long the eyes stay closed and how often the driver yawns, two signs of drowsiness.
• Animal behavior: DeepLabCut tracks user-defined points on lab animals, such as paw and snout positions, and reached human-level accuracy with about 200 labeled frames.