For years, the worry about AI vision was that it makes things up: you ask about a dog that is not in the picture and it describes the dog. In our tests that problem is mostly gone. A different one is not.
Part one: asking about things that are not there
In the “absent objects” test, the question is about an object that may or may not be in the picture. A model that goes along with a false premise gets it wrong. Thirty-two of 36 models score 85 or more out of 100. Among the 24 models tested on our larger sample, Gemini 4 Argon is best at 96.2. Twelve models got every question right: the 12 tested on a smaller sample. Even the small local models do well: Qwen3-VL-8B scores 90, running on one PC.
The “I can’t tell” test is similar. The picture either shows the answer or hides it, and the right reply may be to say so. Twenty-five of 36 models score 85 or better. GPT-6.1 Sol and Gemini 4 Argon score 100, and so do nine models tested on a smaller sample.
US labsChinese labsOpen models via APIRuns locallyOther labs
Each dot is one model. Dots that overlap are nudged up or down.
Part two: optical illusions
In the illusion test, two things in a picture look different but are the same, or look the same but are not, and the model says which is bigger, or that they are equal. Here the picture is the trap.
Eleven of 36 models score 3 or less out of 100. That is no better than picking an answer at random, among three. Eight score slightly below what random guessing would give, though the difference is inside the margin of error. It is what you expect when the trick works: the model reports what the illusion makes it look like.
The best are GPT-6 Astra (81.2), GPT-6.1 Sol (79.6), Claude Sonnet 5.5 (77.5) and GPT-6 Luna (77.2). At the bottom, Gemini 3.8 Flash, DeepSeek V4.1 Flash, Gemini 3.5 Flash-Lite, Qwen3.5-9B and seven others all score 3 or less. All 11 of them were tested on our larger sample.
The same model, two results
The two skills do not go together. Gemini 3.8 Flash scores 95 on absent objects and 0 on illusions. Qwen3.8 Flash scores 92 and 2. DeepSeek V4.1 Flash scores 86 and 0.
Some labs resist illusions and some do not:
- OpenAI. All three models score 77 or more.
- Anthropic. Sonnet scores 77.5, Opus 70, Haiku and Fable 5.2 59, Fable 5.1 44.
- Google. Gemini 4 Argon 40; Gemini 3.8 Flash and Gemini 3.5 Flash-Lite both 0.
- Chinese labs. The six tested on our larger sample all score 18 or below. The other five were tested on a smaller sample. They spread from 10 to 70, too wide to read much into.
Price tells you little. GPT-6 Luna, at $0.74 per 1,000 answers, scores 77, close to GPT-6 Astra’s 81 at $38.32. Claude Fable 5.1, the most expensive model, scores 44.
Why it matters
These are different skills. Declining a false premise is about what a model is willing to claim. Seeing past a visual trick is about what it perceives. Passing the first test says little about the second. If your product relies on a model judging sizes, lengths or brightness in a picture, test it on illusions first.
How we tested
Each test asks a question with one exact answer. The illusion pictures are built so the correct answer differs from the way they look. The absent-object pictures contain, or do not contain, the object asked about. Answers are scored automatically and corrected for guessing, so 0 is a guess and 100 is always right. Twelve models were tested on a smaller sample, so their margins of error are wider; the leaderboard shows them. The pictures and questions are never published.
Tested between 22 October and 10 November 2026. Per-model dates are on the leaderboard.