FAQ
Questions and answers
Short answers about how the index works. The full method is on the method page. Method
What is the AI Vision Index?
It is an independent leaderboard of how well AI models see. We test things like reading clocks, meters, charts and labels, counting, judging space and motion, spotting changes and errors, following cables and pipes, and not being fooled. It ranks 36 models at the moment, with results updated 10 Nov 2026.
How is it different from public benchmarks?
Public benchmarks publish their questions. Over time those questions end up in the data models are trained on, so a high score can just mean the model has seen the answers. We make our own test images and never publish them. Every question has one exact answer that a script checks, so no person or AI has to judge, and every result shows when we tested it.
Why are the tests private?
So that no model can learn them. If we published the pictures, they would soon be copied around the web and end up in training data. Instead we describe each test and show stand-in examples, and the real pictures and questions stay with us.
How do the scores work?
Each test score is the share of exact answers with lucky guesses taken out. 0 means no better than a guess, and 100 means always right. A category score is the average of its tests, and the overall score is the average of the six categories, so no category counts more than another. More on the method page.
Why do some models share a rank?
Every score has a margin of error (a 95% interval). When two models' scores are too close to tell apart, they share a rank, shown as “3–7”. A model run on a smaller sample has a wider interval, so it can tie with more neighbours.
Why do some models cost more per answer?
Makers charge per token (the small pieces of text and image a model reads and writes), and their prices differ a lot. Models that “think” before answering also write many hidden tokens. We report what 1,000 answers cost at each maker's standard list price, thinking included, so you can compare models fairly. Models that run on your own computer cost nothing per answer.
How do you measure speed?
Seconds per answer is the typical (median) time one answer took when we called the model directly at normal priority, or, for models that run locally, on a laptop graphics card. It includes any thinking before the answer. Models we cannot yet time in a comparable way show “–”.
Which models do you test? Can I suggest one?
We test the main models from the big labs, cheaper and smaller ones, and open models that run on a PC. New models go in as they come out. To suggest one, write to us through the Test your images form.
Can I test my own images?
Yes. Send us a few of the pictures you care about and tell us what you need to know from them. We test the models on them privately and tell you how they did. Test your images.
How often are results updated?
Whenever we test a new model or a new version. A model can change while its name stays the same, so every result shows the date we tested it.
Who runs it, and who pays?
We run it independently and pay for it ourselves. No one pays to be listed, and no maker sees the tests before we run them.
Can I reuse or embed the results?
Yes. The aggregate results (scores, intervals, costs and dates) are CC BY 4.0, so you can reuse them if you credit the AI Vision Index and link to it. Download all results as JSON, embed the live top 10 (snippet below), or put a model's badge on your site (on every model page).
How should I cite it?
AI Vision Index (2026). AI Vision Index: independent tests of AI vision models. https://aivisionindex.com/ (results of 10 Nov 2026).
Glossary
- Guessing correction
- The score only counts what a model gets right beyond lucky guesses: (accuracy − chance) ÷ (1 − chance). On a yes-or-no question chance is a half, and on a free answer such as a time or a number it is zero.
- Margin of error (95% interval)
- The range the score would probably fall in if we tested again with other pictures of the same kind. The narrower it is, the surer we are.
- Category
- A group of tests for one kind of seeing: Reading, Counting, Space, 3D and motion, Spotting changes and errors, Diagrams, Not being fooled.
- Overall score
- The average of the six category scores, out of 100.
- Open weights
- The maker publishes the model itself, so anyone can download it and run it on their own computer.
- Thinking effort
- How long a model may reason before it answers, where the maker offers a setting. We note the setting we used.
- Runs locally
- The model ran on our own computer instead of the maker's servers, so each answer costs nothing.
- Cost per 1,000 answers
- What 1,000 answers cost in US dollars at the maker's standard list price, including thinking.
- Seconds per answer
- The typical time one answer took in our runs, shown only where the run reflects normal use.
- Difficulty level
- Every test gets harder in steps, so we can see where a model starts to fail.
Embed and reuse
Put the live top 10 on your page:
<iframe src="https://aivisionindex.com/embed/" title="AI Vision Index top 10" width="100%" height="520" style="border:0" loading="lazy"></iframe>
Every model page has a badge showing its rank and score, and a snippet you can copy. All results (JSON) · The aggregate results (scores, intervals, costs, dates) are CC BY 4.0, so you can reuse them freely if you credit the AI Vision Index and link to it.