How to choose a VLM for auto-annotation using precision and recall

Picking a model to auto-annotate a dataset comes down to the cleanup your reviewers would rather do: delete extra boxes, or add the ones a model missed. The Ultralytics VLM Leaderboard now shows that trade-off for every model, with precision (the share of a model's boxes that are right) and recall (the share of labeled objects it finds), from the benchmark results published October 28, 2026.

For the cleanest first drafts, start with Gemini 4 Argon. To miss the fewest objects, start with GPT-6 Astra. For the best balance, and the lowest cost of the six top-scoring models, choose GPT-6.1 Terra. Without your own AI provider API key, start with the hosted Qwen3.8 27B, or SAM 3.1 Text for crowded scenes you would rather prune.

This guide explains how to read the numbers, where each model fits and how to confirm your choice on your own images. Rankings and prices change after each run, so treat the figures here as October 2026 examples and check the leaderboard for current results.

Class-prompted models have no confidence slider, so the model choice is your threshold#

With an Ultralytics YOLO model in the annotation editor, you tune the trade-off yourself. Raise the confidence threshold and you get fewer false boxes and more misses; lower it and the reverse happens. Class-prompted vision language models (VLMs) work differently. As the annotation editor docs explain, they detect your dataset's classes with model-specific thresholds and return no confidence scores, so Ultralytics Platform hides the confidence and IoU settings while one is selected.

That changes what the leaderboard's precision and recall mean. According to the benchmark methodology, precision and recall count every box a model returns at an IoU of 0.5, with no hidden operating point. They describe the boxes your reviewers actually receive.

The benchmark behind the leaderboard is UL37 v2: 10 images from each of 36 real object detection datasets on Platform, plus 10 COCO 2017 reference images scored separately, for 370 images and 7,268 boxes. Each image runs through the same production pipeline as Smart and Batch Annotation, with the dataset's own class names, and SAM 3.1 refines every model's boxes except SAM 3's. Scoring uses the Ultralytics package's own detection metrics, the code that validates YOLO models, and each dataset counts equally in the average.

Precision tracks unmatched boxes, recall tracks unmatched objects#

Precision is the share of a model's boxes that match a labeled object of the right class at IoU 0.5. Recall is the share of labeled objects the model matched. In review terms:

• Low precision means more boxes that need deleting, relabeling or resizing until they match.
• Low recall means more labeled objects that need a box added, resized or relabeled.
• F1 is the harmonic mean of the two, 2 x P x R / (P + R), one number that sums up both. It doesn't measure how long a review takes.
• mAP₅₀ and mAP_50-95 rank models by average precision across classes, at IoU 0.5 and across the stricter thresholds from 0.50 to 0.95. mAP₅₀ is the leaderboard's sort order.

Here are the six models that score above 0.51 mAP₅₀. Cost is estimated from provider-reported token usage per 1,000 successfully annotated benchmark images, and latency is the median request time:

*Gemini 4 Argon's estimate uses Google's introductory pricing, which runs through November 30, 2026. From December 1, Google's standard rate doubles, to about $15 per 1,000 images at the same usage. Batch Annotation on Ultralytics Platform adds $1.00 per 1,000 images processed for any model, hosted ones included.

The six split into two camps. Argon, Opus 5.5 and Gemini 3.8 Flash sit at 0.66 to 0.67 precision with recall below 0.60: a high share of their boxes match, but they match fewer of the labeled objects. Astra and Sol reach 0.62 to 0.63 recall with precision of 0.55 to 0.56: they match a higher share of labeled objects, while a lower share of their boxes match. Terra sits between the camps and leads on mAP₅₀ and F1.

Prioritize precision when stray boxes are the costly fix#

At 0.667, Gemini 4 Argon has the highest overall precision of the 40 models on the benchmark. Its trade-off is an average recall of 0.585, below the recall leaders.

Precision matters most when unmatched boxes are hard to spot. On a crowded image, a reviewer has to find a bad box among many good ones before fixing it. The dense retail shelves of SKU110K show how far precision can swing on such images: in our GPT-6.1 Terra results, Terra recorded 0.79 precision and 0.73 recall there, against GPT-6.1 Sol's 0.49 and 0.77.

Gemini 4 Argon: 5 boxes predicted for 4 labeled objects, precision 0.60, recall 0.75.

Argon also had no failed requests on the benchmark, and its 4.6-second median is the quickest of the top two. Choose it when your data resembles its domain wins, aerial, underwater, thermal and security X-ray imagery (it posts the best mAP₅₀ on the IEDXray security scans, 0.298), and when your team would rather complete a few missed objects than hunt for stray boxes.

Prioritize recall when a missed object is the costly error#

GPT-6 Astra has the highest average recall on the board, 0.627. Its precision of 0.56 is the price.

Recall-first labeling makes sense when an unlabeled object is the costlier error. A missed defect or lesion in your training set teaches the detector that the object is background, and finding a missing box means scanning every image for something that isn't marked. Astra posts the best mAP₅₀ on 7 of the 36 datasets, including TBX11K chest X-rays (0.330), GRAZPEDWRI-DX pediatric wrist X-rays (0.354) and DsPCBSD+ PCB surface defects (0.394): a reason to test it on similar data, not proof it misses less on yours.

GPT-6 Astra: 6 boxes for the same 4 objects, including an umbrella the COCO labels leave out, precision 0.50, recall 0.75.

Astra has the highest cost estimate in the top six, $49.31 per 1,000 images, with an 11.6-second median and failed requests on 1% of images. Two other OpenAI models come close on recall: GPT-6.1 Sol at 0.62, estimated at $10.30, and GPT-6.1 Terra at 0.61, estimated at $4.36.

Weigh both errors with mAP_50-95 and F1#

GPT-6.1 Terra tops the board at 0.560 mAP₅₀ and 0.345 mAP_50-95. Its 0.646 precision and 0.613 recall are both within a few hundredths of the leaders, and its F1 of 0.63 is the highest of the six, just ahead of Argon and Opus 5.5 at 0.62. With F1 that close, and F1 saying nothing about review time, let the per-domain results and a test on your own images decide between them.

Its mAP_50-95 lead adds a second signal. Because mAP_50-95 averages over overlap thresholds up to 0.95, it rewards models whose matched boxes fit closely as well as models that find the right objects, and Terra is first on it too. It also has the lowest cost estimate in the top six and had no failed requests, and the leaderboard marks it "Best value."

Label without a provider key: hosted models and open-vocabulary detectors#

Not every team wants a paid provider key. The hosted models on Platform run without one, and their columns tell a different story:

• SAM 3.1 Text: 0.332 mAP₅₀, 0.19 precision, 0.47 recall, 0.2 s per image. It is the best model of all 40 on VisDrone (0.479 mAP₅₀) and Fisheye8K traffic (0.478).
• Qwen3.8 27B: 0.351 mAP₅₀, 0.54 precision, 0.40 recall, 1.5 s.
• Gemma 4 26B: 0.344 mAP₅₀, 0.53 precision, 0.39 recall, 0.9 s.

The open-vocabulary detectors are hosted too. In this benchmark, OWLv2 Large scored 0.133 mAP₅₀ and Grounding DINO Base scored 0.104, both at 0.2 seconds per image, against SAM 3.1 Text's 0.332. These scores measure Platform's class-prompted pipeline, with SAM 3.1 box refinement, not each model's original setup.

SAM 3.1 Text: 29 boxes for 4 objects, precision 0.10, recall 0.75. All three replays find 3 of the 4 objects; what changes is how many extra boxes reviewers delete.

SAM 3.1 Text suits crowded aerial and traffic scenes where reviewers expect to prune a generous draft. Qwen3.8 27B and Gemma 4 26B behave like the careful camp at lower accuracy. Among low-cost paid models, Claude Haiku 5.5 reaches 0.429 mAP₅₀ with 0.56 precision and 0.47 recall, estimated at $0.37 per 1,000 images, the other model the leaderboard marks "Best value."

Board-wide averages hide your domain, so check the dataset table#

Each model's precision and recall average 36 datasets, and your data is one domain. The VLM leaderboard's dataset table lists every dataset's mean precision and recall across all 40 models, hardest first, plus the best model on each.

Read it for warning signs. On Airbus Oil Storage Detection, with 2,267 labeled tanks across its 10 images, the models average 0.39 precision and 0.11 recall, and the best mAP₅₀ is Gemini 4 Argon's 0.347. On DeepPCB, mean precision and recall are 0.03 and 0.04, and GPT-6.1 Sol stands out with 0.393 mAP₅₀.

Treat these numbers as a shortlist, not a verdict. The methodology states that 10 images per dataset are enough to rank models and domains but not to certify a single dataset's score, and repeated runs move individual numbers by a few points. Each model page on Platform replays the model's recorded boxes beside the expected ones on the benchmark datasets and COCO, so you can inspect the dataset closest to yours before you start a run.

Match the model to your review workflow in five steps#

Start from your selection priority, then confirm the choice on your own images:

Class-prompted annotation works on detection datasets with 1 to 200 classes.

1. Shortlist on the leaderboard. Filter the VLM leaderboard for object detection to Paid or Hosted models (skip any marked Not on Platform), read the column that matches your priority, and check the dataset table for your domain.
2. Add a provider key. For a paid model, save your key under Settings > API Keys in the workspace that owns the dataset. The provider bills your key directly; the provider key docs cover setup.
3. Test on a reviewed sample. Pick a few images that represent your data and that you have labeled by hand. Open the annotation editor, switch to Smart mode, pick the model under its provider in the model picker, and press Predict (P). Count the boxes you delete, the boxes you add and the minutes the review takes.
4. Label the dataset. On the dataset page, choose More actions > Batch Annotation, available on Platform-hosted datasets with up to three image channels. It labels unlabeled images by default, never changes existing labels, and saves a dataset version first so you can roll back. Review the estimated cost in the dialog before you start: Batch Annotation costs $1.00 per 1,000 images processed, plus any provider charges.
5. Review, then train. Check the boxes with your model's weak side in mind, save them, and train an Ultralytics YOLO model on the result in the same workspace.

For background, see how Ultralytics Platform uses AI to automate annotation and why human-in-the-loop review stays part of the job. To see how auto-labeling compares with active learning across tools, read our auto-labeling and active learning comparison.

Shortlist by team (October 2026 results)#

• Reviewers who would rather complete a draft than clean one up: Gemini 4 Argon, the most precise model overall, strongest on aerial, underwater, thermal and security X-ray data.
• Rare objects where a miss poisons training data: GPT-6 Astra for maximum recall, or GPT-6.1 Sol close behind.
• General-purpose labeling with no strong preference: GPT-6.1 Terra, first on mAP₅₀, mAP_50-95 and F1.
• Teams without a provider key: SAM 3.1 Text for recall-heavy first passes on crowded scenes, or Qwen3.8 27B for a more careful draft.

Whichever you pick, the leaderboard lets you anticipate the review before you start it. For the theory behind these metrics, our guide to accuracy, precision and recall covers the basics, and the Academy lesson on reading detection metrics goes further.