Skip to content
knowledge benchmark · included in ECI

SimpleQA Verified

Short factual questions testing parametric knowledge and hallucination.
Epoch AI source ↗
Results
78
Random baseline
0.0%
Score ceiling
100%
Released
9 Sep 2025

Score by model release date

Best score by organisation

Leaderboard

Open weights only
#ModelScoreRelativeEvidence
1GPT-6 Astra
OpenAI
75.6% ±1.4Source ↗
2Gemini 3.1 Pro Preview
Google
73.5% ±1.4Source ↗
3Claude Fable 5.1
Anthropic
70.8% ±1.4Source ↗
4Claude Fable 5 (xhigh)
Anthropic
70.7% ±1.4Source ↗
5Gemini 3.8 Flash
Google
69.7% ±1.5Source ↗
6GPT-5.6 Sol
OpenAI
69.7% ±1.5Source ↗
7Gemini 3.7 Flash
Google
69.2% ±1.5Log ↗
8Gemini 3 Flash Preview
Google
66.8% ±1.5Log ↗
9Gemini 3.6 Flash
Google
66.2% ±1.5Log ↗
10Gemini 3.5 Flash
Google
66.2% ±1.5Log ↗
11GPT-5.5 (xhigh)
OpenAI
63.0% ±1.5Log ↗
12Muse Spark 1.2 (xhigh)
Meta AI
60.3% ±1.6Log ↗
13Claude Opus 5
Anthropic
59.9% ±1.6Source ↗
14Muse Spark 1.1
Meta
57.8% ±1.6Log ↗
15Qwen3.7 Max
Qwen
55.8% ±1.6Log ↗
16Claude Opus 4.8
Anthropic
53.0% ±1.6Log ↗
17DeepSeek V4 Pro 0813 Open source
DeepSeek
52.9% ±1.6Log ↗
18Qwen 3.6 Max (Preview)
Alibaba
52.0% ±1.6Log ↗
19Claude Opus 4.7 (xhigh)
Anthropic
51.7% ±1.6Log ↗
20Kimi K3 Open source
MoonshotAI
50.6% ±1.6Log ↗
21GPT-5
OpenAI
50.1% ±1.6Log ↗
22o3
OpenAI
49.4% ±1.6Log ↗
23Grok 4.6
SpaceXAI
49.3% ±1.6Log ↗
24Grok 4.6 (xhigh)
xAI
48.9% ±1.6Log ↗
25Qwen3-Max-Instruct
Alibaba
48.7% ±1.6Log ↗
26Grok 4.5
SpaceXAI
48.3% ±1.6Log ↗
27GPT-5.1
OpenAI
48.0% ±1.6Log ↗
28Qwen3.8 Max (0902) (xhigh)
Alibaba
47.3% ±1.6Source ↗
29Claude Opus 4.6
Anthropic
47.0% ±1.6Log ↗
30DeepSeek v4 Pro (high) Open source
DeepSeek
47.0% ±1.6Log ↗
31GPT-5.4 Pro (xhigh)
OpenAI
46.3% ±1.6Log ↗
32Qwen3.8 Max (xhigh)
Alibaba
45.8% ±1.6Log ↗
33Claude Opus 4.5 (32k thinking)
Anthropic
45.7% ±1.6Log ↗
34GPT-5.4 (xhigh)
OpenAI
45.1% ±1.6Log ↗
35Qwen 3.6 Plus (2026-04-02)
Alibaba
44.1% ±1.6Log ↗
36GPT-5.6 Terra
OpenAI
43.2% ±1.6Source ↗
37o1
OpenAI
41.1% ±1.6Log ↗
38GLM 5.3 Open source
Z.ai
41.0% ±1.6Log ↗
39GPT-5.6 Luna
OpenAI
41.0% ±1.6Source ↗
40Qwen3-235B-A22B-Thinking (Jul 2025) Open source
Alibaba
40.4% ±1.6Log ↗
41Inkling (xhigh) Open source
Thinking Machines
40.3% ±1.6Log ↗
42GPT-5.2 (xhigh)
OpenAI
37.1% ±1.5Log ↗
43Kimi K2.7 Code Open source
MoonshotAI
36.5% ±1.5Log ↗
44Claude Sonnet 4.6
Anthropic
35.5% ±1.5Source ↗
45Kimi K2.6 Open source
MoonshotAI
34.9% ±1.5Source ↗
46Kimi K2.5 Open source
MoonshotAI
34.3% ±1.5Log ↗
47GPT-5.2
OpenAI
34.3% ±1.5Log ↗
48GLM 5.2 Open source
Z.ai
34.2% ±1.5Log ↗
49GLM 5.1 Open source
Z.ai
34.0% ±1.5Log ↗
50Claude Sonnet 5
Anthropic
33.7% ±1.5Log ↗
51DeepSeek V4 Flash 0731 Open source
DeepSeek
33.6% ±1.5Log ↗
52Grok 4.3 Beta
xAI
33.2% ±1.5Log ↗
53Claude Sonnet 5 (xhigh)
Anthropic
32.9% ±1.5Log ↗
54GPT-5.2 (low)
OpenAI
32.8% ±1.5Log ↗
55GPT-5.2 (medium)
OpenAI
32.7% ±1.5Log ↗
56GLM 4.7 Open source
Z.ai
32.2% ±1.5Log ↗
57GPT-4.1
OpenAI
31.1% ±1.5Log ↗
58Claude Sonnet 4.5 (59k thinking)
Anthropic
30.7% ±1.5Log ↗
59Grok 4.20
SpaceXAI
30.2% ±1.5Log ↗
60GPT-5.4 Mini
OpenAI
29.4% ±1.4Log ↗
61GPT-4o (Aug 2024)
OpenAI
26.0% ±1.4Log ↗
62Qwen3.5 Plus 2026-02-15
Qwen
25.4% ±1.4Log ↗
63Claude Sonnet 4.5 (no thinking)
Anthropic
23.7% ±1.3Source ↗
64GPT-5 Mini
OpenAI
21.6% ±1.3Source ↗
65Qwen3.5-Flash
Qwen
20.3% ±1.3Log ↗
66o4-mini (low)
OpenAI
19.6% ±1.3Log ↗
67Inkling Small (xhigh) Open source
Thinking Machines
19.1% ±1.2Log ↗
68o4 Mini
OpenAI
18.8% ±1.2Log ↗
69Qwen 3.6 Flash (2026-04-16)
Alibaba
15.9% ±1.2Log ↗
70o3 Mini
OpenAI
15.3% ±1.1Log ↗
71Claude Haiku 4.5
Anthropic
13.2% ±1.1Source ↗
72GPT-4.1 Mini
OpenAI
12.7% ±1.1Log ↗
73Claude 3 Opus
Anthropic
12.6% ±1.0Log ↗
74GPT-5.4 Nano
OpenAI
11.7% ±1.0Log ↗
75GPT-5 Nano
OpenAI
11.7% ±1.0Source ↗
76Gemma 4 31B Open source
Google
10.4% ±1.0Log ↗
77GPT-4o-mini
OpenAI
8.3% ±0.9Log ↗
78GPT-4.1 Nano
OpenAI
6.0% ±0.8Log ↗

Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.