“How many are there?” sounds like the easiest question you could ask an AI that looks at pictures. Stock on a shelf, people at a door, boxes on a pallet: a lot of real work is counting. We asked all 36 models in the index to count, and we know the exact answer to every picture because we built the pictures ourselves.
The short answer: AI counts small numbers well, and loses count fast once a scene gets busy.
Up to 10, fine. Past 20, mostly wrong.
In our “Counting objects” test, a model sees a cluttered scene and must give the exact number of one kind of object. We grouped the pictures by how many objects there really are:
| Objects in the picture | Best model | Share counted exactly right | Average of the models shown |
|---|---|---|---|
| Up to 10 | Gemini 4 Argon, GPT-6 Astra, Claude Fable 5.2 | 100% | 65% |
| 11 to 20 | Claude Fable 5.2 | 67% | 31% |
| 21 to 30 | GPT-6.1 Sol | 21% | 6% |
| More than 50 | GPT-6 Astra | 12% | 2% |
Thin grey lines: one per model. Bold: the two best counters and the average of all models shown. Only an exact number counts as right.
The table and chart leave out the 12 models we ran on a smaller sample, because they have too few pictures in each group. Of the models shown, five count right nine times in ten or more with up to 10 objects. Between 11 and 20, the best, Claude Fable 5.2, gets two in three. Between 21 and 30, the best model, GPT-6.1 Sol, is right one time in five, and most models almost never. Above 50, nearly every answer is wrong.
Wrong, but how wrong?
An exact count is a hard bar, so we also looked at how far off the wrong answers are. On the busiest scenes, the two best OpenAI models are close: half of GPT-6 Astra’s answers are within 8% of the true number, and half of GPT-6.1 Sol’s within 10%. Claude Opus 5.5, Claude Sonnet 5.5 and Claude Fable 5.2 are typically about a third off. Three of the five small models that run on a PC are typically out by two thirds.
So the leaders estimate well and count badly. That is fine for “roughly how many people came?” and not fine for “how many items are missing from this order?”.
Clutter is the problem, not numbers
The same models count far better when the objects are orderly or arrive one at a time.
- Stacked goods. On pallets of boxes, GPT-6.1 Sol still counts 21 to 50 boxes exactly right about three times in four, and GPT-6 Astra nearly as often. With 31 to 50 boxes, no other model tested on our larger sample gets even half right.
- A video conveyor. When items pass by one at a time, GPT-6 Astra counts every clip right, and GPT-6.1 Sol nearly every one.
What breaks counting is many similar things, close together, seen at once.
US labsChinese labsOpen models via APIRuns locallyOther labs
The figure at the end of each bar is the score, then the cost per 1,000 answers.
The counting league table
Counting is one of our six categories (busy scenes, stacked goods and the conveyor video). It is the weakest category for 26 of the 36 models. GPT-6 Astra (78) and GPT-6.1 Sol (76) lead it, clear of every model tested on a larger sample; the next best are Claude Fable 5.2 (64), Claude Fable 5.1 and Claude Opus 5.5 (63). Models tested on a smaller sample have wider margins. Qwen3.8 Max from Alibaba scores 56, with an interval from 40 to 69, so it may be as good as those three, or well behind them. Kimi K3.1 Pro, also on a smaller sample, scores 53.
What it means
If your product counts, test it on your own pictures at the counts you expect. Up to about ten objects, a top model will usually be right. In a crowded scene, treat its number as an estimate and check it, or use a dedicated counting tool. If you can control the scene, spread things out or count them one at a time: the same models do far better.
How we tested
Every counting picture is built by script or in 3D, so the true number is known exactly. Scenes get harder step by step, with more objects, more clutter and more look-alikes. The model must answer with a single number; only an exact match counts as right. Here we grouped the answers by the true number of objects, which is not how the leaderboard scores the test (it averages the difficulty levels), so the shares above are not the leaderboard scores. The pictures are never published.
Tested 22 October to 10 November 2026. Each model’s dates are on the leaderboard.