By test

Where each model is strong, and where it slips

The 29 tests fall into six categories. Each score, out of 100, is the share of exact answers with lucky guesses taken out. Hover over or tap a cell for its 95% interval.

Fig. 1Score on each test, out of 100, grouped by category

Score out of 100 on each of the 29 tests, grouped into 6 categories
	Reading								Counting			Space, 3D and motion							Spotting changes and errors				Diagrams			Not being fooled

GPT-6 Astramedium	98	98	98	78	91	75	80	89	52	80	100	60	85	75	85	87	97	97	92	88	55	100	92	100	100	94	96	81	15
GPT-6.1 Solmedium	98	91	88	78	80	70	74	80	50	81	98	80	73	61	74	84	97	92	87	85	53	99	91	98	98	89	100	80	19
Claude Fable 5.2default	95	95	52	90	75	78	70	96	52	42	98	80	82	75	70	93	100	100	82	100	64	97	90	100	95	94	96	59	40
Gemini 4 Argondefault	96	90	82	89	58	72	66	84	37	54	83	90	97	24	61	82	98	52	94	94	73	94	85	95	91	96	100	40	61
Claude Opus 5.5default	91	85	44	88	76	70	67	78	43	52	92	70	79	20	59	82	100	88	74	95	67	100	81	98	92	93	97	70	44
Claude Sonnet 5.5default	87	90	32	85	63	72	61	64	46	60	53	65	74	55	40	74	97	70	90	72	52	81	78	85	90	95	94	78	44
Kimi K3.1 Prodefault	80	87	60	87	67	47	60	90	47	20	93	100	69	0	40	56	100	72	87	87	36	100	87	87	100	100	100	50	20
Qwen3.8 Maxdefault	87	73	40	87	42	80	60	70	53	53	60	87	69	33	70	100	93	72	93	73	36	86	60	93	100	100	100	30	47
Claude Fable 5.1default	65	82	25	85	56	72	52	62	42	60	87	75	59	40	55	61	100	92	60	90	52	100	63	75	90	94	99	44	48
Claude Haiku 5.5default	81	76	31	78	50	70	40	74	29	37	78	44	63	9	35	54	92	35	76	84	32	97	76	84	76	95	96	59	39
Gemini 3.8 Flashmedium	72	51	48	82	48	38	53	75	30	54	41	75	76	26	62	72	93	56	80	92	67	99	58	65	81	95	97	0	20
GPT-6 Lunamedium	75	70	40	66	46	62	44	70	38	33	33	65	49	8	51	65	92	47	77	78	44	91	69	58	86	90	99	77	24
Qwen3.8-27Bdefault	60	40	20	60	50	80	40	70	80	0	60	100	29	0	40	47	100	45	80	80	52	100	40	60	80	100	100	40	20
Qwen3.8 Flashdefault	65	55	38	77	46	66	49	68	34	27	61	73	59	0	51	55	96	53	74	68	25	89	57	54	87	92	92	2	35
Grok 4.7default	40	80	0	100	50	60	60	40	60	60	40	100	29	20	10	47	100	45	60	60	52	100	40	0	80	100	67	70	20
Mistral Large 4default	40	60	20	80	50	60	60	10	100	0	20	60	29	0	10	47	100	17	80	60	28	100	60	60	80	100	100	10	20
Muse Spark (open)default	60	80	20	80	50	60	40	40	60	20	20	60	53	0	10	47	100	17	40	40	52	79	40	20	60	100	100	40	20
GLM-5V-Turbodefault	70	48	12	60	41	50	50	70	22	25	10	70	47	0	40	53	89	42	52	60	52	87	48	32	60	94	96	14	10
DeepSeek V4.1 Flashdefault	26	36	20	68	50	45	32	48	29	2	22	65	36	0	24	42	89	37	62	58	30	92	52	32	78	86	90	0	31
GLM-5.3 Flashdefault	42	43	26	61	44	52	25	46	26	11	12	54	45	0	18	53	75	30	56	58	27	82	40	37	70	89	90	11	36
Qwen3.6-35B-A3Bdefault	80	80	20	80	50	60	60	70	40	0	0	20	53	0	10	47	78	17	60	60	28	36	40	40	60	100	100	10	20
Qwen3-VL-235Bdefault	60	60	20	100	50	20	20	40	20	40	0	60	29	0	0	47	78	25	100	40	28	0	60	40	20	100	100	40	20
MiniMax M3default	32	35	22	60	41	45	40	51	12	8	25	60	53	0	14	47	69	17	60	55	34	68	50	32	72	94	71	18	5
Step-5 (preview)default	40	40	0	60	75	60	20	40	20	40	0	60	29	0	10	0	78	50	40	60	28	57	40	40	60	100	67	40	20
MiMo v2.6 Prodefault	9	32	8	69	37	49	32	52	18	23	17	66	29	0	12	32	88	28	55	71	19	87	51	32	54	92	79	8	5
Gemini 3.5 Flash-Litedefault	23	38	4	80	53	49	20	51	16	30	18	47	42	0	32	54	76	24	32	50	39	55	50	19	62	86	89	0	37
Gemma 4 31Bdefault	20	40	20	60	50	40	20	40	0	0	0	20	6	0	70	0	78	45	80	20	76	15	40	40	40	100	100	10	100
Seed 2.1 Turbodefault	0	40	40	60	25	20	20	10	80	0	0	20	6	0	10	0	33	45	80	40	28	36	40	20	80	100	100	10	60
Hunyuan HY4default	40	40	40	60	25	0	20	40	0	0	20	20	29	0	0	20	55	45	20	60	52	79	40	20	80	100	67	70	20
Qwen3.5-9Bno thinking	50	38	3	68	47	33	4	50	19	10	28	60	23	13	10	50	31	0	48	8	31	8	35	6	34	89	73	0	20
Perceptron Mk1.5default	51	45	1	69	51	41	4	54	14	8	—	62	27	0	23	56	47	—	32	18	7	6	34	14	23	91	76	0	21
Qwen3-VL-8Bdefault	41	27	0	72	46	30	5	41	11	18	23	65	21	0	1	38	14	17	36	11	23	4	32	10	14	90	86	0	9
MiniCPM-V 4.5default	2	16	0	55	31	27	0	38	10	14	13	30	9	0	0	0	0	5	38	8	12	6	18	2	1	82	62	3	12
Gemma 4 E4Bdefault	0	10	0	54	26	26	4	12	9	0	0	44	8	0	1	10	5	0	12	10	5	2	10	6	4	67	63	0	27
Mistral Small 2603default	0	6	0	61	29	22	2	0	6	14	—	26	0	0	7	6	0	—	32	4	6	6	10	2	16	38	50	0	0
LFM2.5-VL 1.6Bdefault	2	6	0	41	37	24	0	34	11	2	4	12	0	5	1	1	0	0	16	8	30	5	7	3	2	55	55	0	5

Darker cells mean higher scores, and — means the model couldn’t take that test. Click a column name to sort. Scroll sideways for all 29 tests.

Video

Video understanding

Two of our tests are short clips. In one, the model follows a hidden object as containers move; in the other, it counts items going past on a conveyor belt. Both test whether it can keep track of things over time.

A model sees a video as a series of still pictures. To answer, it has to recognise the same thing from one frame to the next, remember where it went while it was hidden and avoid counting anything twice. Most models learn mostly from single pictures, so following a whole clip is harder for them than reading one image.

Some models accept only a handful of pictures per question, so we can’t give them these tests and they have no score here.

Follow the hidden object

Containers slide around and one of them hides an object. The model has to say where it ended up.

Counting on a conveyor

Items ride past on a belt. The model has to say how many went by, counting each one once.

41average on the two video tests52average on all other tests

Fig. 2Score on the two video tests, out of 100

ModelFollow the hidden objectCounting on a conveyor

Claude Fable 5.2default

Follow the hidden object100

Counting on a conveyor98

GPT-6 Astramedium

Follow the hidden object97

Counting on a conveyor100

GPT-6.1 Solmedium

Follow the hidden object92

Counting on a conveyor98

Claude Opus 5.5default

Follow the hidden object88

Counting on a conveyor92

Claude Fable 5.1default

Follow the hidden object92

Counting on a conveyor87

Kimi K3.1 Prodefault

Follow the hidden object72

Counting on a conveyor93

Gemini 4 Argondefault

Follow the hidden object52

Counting on a conveyor83

Qwen3.8 Maxdefault

Follow the hidden object72

Counting on a conveyor60

Claude Sonnet 5.5default

Follow the hidden object70

Counting on a conveyor53

Qwen3.8 Flashdefault

Follow the hidden object53

Counting on a conveyor61

Claude Haiku 5.5default

Follow the hidden object35

Counting on a conveyor78

Qwen3.8-27Bdefault

Follow the hidden object45

Counting on a conveyor60

Gemini 3.8 Flashmedium

Follow the hidden object56

Counting on a conveyor41

Grok 4.7default

Follow the hidden object45

Counting on a conveyor40

GPT-6 Lunamedium

Follow the hidden object47

Counting on a conveyor33

Hunyuan HY4default

Follow the hidden object45

Counting on a conveyor20

DeepSeek V4.1 Flashdefault

Follow the hidden object37

Counting on a conveyor22

GLM-5V-Turbodefault

Follow the hidden object42

Counting on a conveyor10

Step-5 (preview)default

Follow the hidden object50

Counting on a conveyor0

Gemma 4 31Bdefault

Follow the hidden object45

Counting on a conveyor0

Seed 2.1 Turbodefault

Follow the hidden object45

Counting on a conveyor0

MiMo v2.6 Prodefault

Follow the hidden object28

Counting on a conveyor17

MiniMax M3default

Follow the hidden object17

Counting on a conveyor25

GLM-5.3 Flashdefault

Follow the hidden object30

Counting on a conveyor12

Gemini 3.5 Flash-Litedefault

Follow the hidden object24

Counting on a conveyor18

Qwen3-VL-8Bdefault

Follow the hidden object17

Counting on a conveyor23

Mistral Large 4default

Follow the hidden object17

Counting on a conveyor20

Muse Spark (open)default

Follow the hidden object17

Counting on a conveyor20

Qwen3.5-9Bno thinking

Follow the hidden object0

Counting on a conveyor28

Qwen3-VL-235Bdefault

Follow the hidden object25

Counting on a conveyor0

MiniCPM-V 4.5default

Follow the hidden object5

Counting on a conveyor13

Qwen3.6-35B-A3Bdefault

Follow the hidden object17

Counting on a conveyor0

LFM2.5-VL 1.6Bdefault

Follow the hidden object0

Counting on a conveyor4

Gemma 4 E4Bdefault

Follow the hidden object0

Counting on a conveyor0

Perceptron Mk1.5default

Follow the hidden objectNot supported

Counting on a conveyorNot supported

Mistral Small 2603default

Follow the hidden objectNot supported

Counting on a conveyorNot supported

Ordered by the average of the two. “Not supported” means the model accepts fewer pictures per question than a video test sends.