Running an AI model on your own machine is free, private and works offline. It is also tempting for anything that looks at personal photos, factory cameras or a phone app. So how much eyesight do you give up?
A lot. We ran five open-weight vision models on one consumer PC and gave them the same private tests as the cloud models. The best, Qwen3.5-9B from Alibaba, scores 29.5 out of 100. That is 35% of the top score on the index, 83.7 for GPT-6 Astra.
Runs on our own graphics cardHosted by its maker
Overall score out of 100 with its 95% interval, then the cost per 1,000 answers.
The five local models
| Model | Maker | Score |
|---|---|---|
| Qwen3.5-9B | Alibaba | 29.5 |
| Qwen3-VL-8B | Alibaba | 26.0 |
| MiniCPM-V 4.5 | OpenBMB | 17.1 |
| Gemma 4 E4B | 13.7 | |
| LFM2.5-VL 1.6B | Liquid | 12.4 |
For scale: Gemini 3.5 Flash-Lite, Google’s small cloud model, scores 40.3, and costs 34 cents per 1,000 answers. That is about a thirtieth of a cent per question. It beats the best local model by 10.8 points. GPT-6 Luna scores 60.7, twice as much as the best local model, for 74 cents. Free does not mean good value if you need the answer to be right.
What local models can do
They are not useless. Two things work:
- Saying whether something is in the picture. Qwen3-VL-8B scores 90 on questions about objects that may be absent, and Qwen3.5-9B 89. The best cloud models score 95 to 100.
- Simple reading and judging distance. Qwen3-VL-8B scores 72 on reading a code from a label and 65 on which object is nearer.
What they cannot do
Averaged over the five local models, the category scores are 12 for Counting, 12 for Diagrams, 14 for Space, 16 for Spotting changes and 25 for Reading. The 13 cloud models from US labs average 53, 75, 64, 74 and 67 on the same categories. The least bad is Not being fooled (40 against 69), carried by the easy “is it there?” questions. On optical illusions all five score 3 or less. On video, Gemma 4 E4B scores 0 on both tests.
Bigger open models do better
Open weights are not only for small models. Four bigger ones are in the index, from 27 to 235 billion parameters. They need more memory than a typical PC has, so we ran them through an API. They were tested on a smaller sample, so their margins are wider:
| Model | Maker | Score | 95% interval |
|---|---|---|---|
| Qwen3.8-27B | Alibaba | 59.0 | 52 to 68 |
| Qwen3.6-35B-A3B | Alibaba | 43.0 | 37 to 52 |
| Qwen3-VL-235B | Alibaba | 41.2 | 35 to 50 |
| Gemma 4 31B | 38.8 | 33 to 47 |
All four beat every local model, even at the low end of their interval. Qwen3.8-27B scores twice as much as the best local model, level with GPT-6 Luna within the margin. Kimi K3.1 Pro from Moonshot also has published weights and scores 70.6, but it needs far more than one PC.
Bigger is not always better
Qwen3.8-27B (59.0) beats Qwen3-VL-235B (41.2), a model almost nine times its size, and their intervals do not overlap. Among the local models, Qwen3.5-9B (29.5) beats Qwen3-VL-8B (26.0), and the smallest, LFM2.5-VL 1.6B (12.4), is not far behind Gemma 4 E4B (13.7). Within one family, size does help: Gemma 4 31B scores 38.8, almost three times its small sibling.
What this means
For private, offline work where an occasional wrong answer is fine, a local model can screen pictures and answer “is there a …?” questions. With a strong graphics card or a server, a bigger open model such as Qwen3.8-27B reaches the level of a cheap cloud model. For anything that must be right, and above all counting, reading dials or following lines, use a top cloud model.
How we tested
Local models ran in LM Studio on one PC, in the compressed form that fits in memory. We used each model’s default settings, except Qwen3.5-9B, which had its thinking turned off. Every model got the same questions in the same format, and answers are scored automatically against an exact key, then corrected for guessing. Some models, these four among them, were tested on a smaller sample, so their margins are wider; the leaderboard shows each one.
Tested 22 October to 10 November 2026. See the leaderboard.