What we test
We run 29 tests of everyday seeing, grouped into six categories. Each test gets harder in steps, so we can see where a model starts to fail instead of giving it a single grade.
- ReadingTelling the time, reading meters, charts, labels, codes and signals.
Reading a clock · Reading a meter · Eye chart · Reading a label · Reading a chart · Test strips · Adding up coins · Traffic light from a car - CountingHow many things there are, in a picture or a video.
Counting objects · Counting stacked goods · Counting on a conveyor - Space, 3D and motionDepth, direction, shape, balance and following things as they move.
Which is nearer · Which way it faces · Same or mirror image · Will it topple · Match the render · Which bay can I park in? · Follow the hidden object - Spotting changes and errorsWhat changed, what is wrong, what does not match.
Spot the change · Shelf against plan · Rendering glitch · Check the invoice - DiagramsFollowing cables and pipes and reading technical drawings.
Follow the cable · Follow the pipe · Missing dimension - Not being fooledAbsent objects, illusions, fakes, and saying "I can't tell" when it is true.
Absent objects · Saying "I can't tell" · Optical illusions · Real or AI-made
Our test images
We make our own test images and never publish them, so no model can have trained on them. Because we built each picture, we know its right answer exactly. We can also make a test harder one step at a time, with smaller letters, more objects or a tilted dial.
A person checks every item. If the picture is unclear or the answer is open to argument, we drop the item and it never counts.
How we score
Every question asks for a short, exact answer, such as a time, a number, a code or a pick from a list. A script compares it with the right answer, and a few tests accept answers within a stated margin. No AI judge or human grader is involved. Within a test, every difficulty step counts the same.
Luck helps on multiple choice. With two choices, a blind guess is right half the time, so we take that guess level off.
what guessing alone scoreswhat the model gained on top
A test’s guess level is 1 divided by the number of choices, averaged over its items, and 0 for free answers. You can find it for every test in leaderboard.json, next to the raw accuracy.
Some categories have more tests than others. We average the categories rather than the tests, so one big category can’t decide the overall score.
Intervals and ties
Each score comes with a 95% interval. To get it, we resample the answers many times, recompute the whole score each time and keep the middle 95% of the results. The overall and category intervals come from the same resamples, so they reflect how the tests combine.
When one model’s overall score sits inside the interval of the model next to it, or the other way round, the two share a rank, shown as “2–3”.
Tests a model cannot take
Two tests are videos, which we send as a series of pictures. Some models accept only a handful of pictures per question, so they can’t take these tests. They get no score there (shown as —), and their category and overall scores average the tests they did take.
At the moment this applies to Perceptron Mk1.5 and Mistral Small 2603.
Dates
A model can change while keeping its name, so every result shows the date of its most recent answer.
What is private and what is public
- Private: the test images, their questions and right answers, and every model’s individual answers.
- Public: the scores, their intervals, raw accuracy and guess level per test, the cost per 1,000 answers (at each provider’s standard price), and the test date. All of it is in leaderboard.json.
Cite us
Please cite the AI Vision Index and link to the page you used. Scores change as models and tests are added, so say when you looked.
AI Vision Index (2026). Independent, private tests of how well AI models see. https://aivisionindex.com/ (accessed 10 Nov 2026).
This is our first public edition, and we are still adding items, tests and models.