Skip to content
knowledge benchmark · included in ECI

SimpleQA Verified

Short factual questions testing parametric knowledge and hallucination.
Epoch AI source ↗
Results
15
Random baseline
0.0%
Score ceiling
100%
Released
9 Sep 2025

Score by model release date

Best score by organisation

Leaderboard

Open weights only
#ModelScoreRelativeEvidence
1DeepSeek V4 Pro 0813 Open source
DeepSeek
52.9% ±1.6Log ↗
2Kimi K3 Open source
MoonshotAI
50.6% ±1.6Log ↗
3DeepSeek v4 Pro (high) Open source
DeepSeek
47.0% ±1.6Log ↗
4GLM 5.3 Open source
Z.ai
41.0% ±1.6Log ↗
5Qwen3-235B-A22B-Thinking (Jul 2025) Open source
Alibaba
40.4% ±1.6Log ↗
6Inkling (xhigh) Open source
Thinking Machines
40.3% ±1.6Log ↗
7Kimi K2.7 Code Open source
MoonshotAI
36.5% ±1.5Log ↗
8Kimi K2.6 Open source
MoonshotAI
34.9% ±1.5Source ↗
9Kimi K2.5 Open source
MoonshotAI
34.3% ±1.5Log ↗
10GLM 5.2 Open source
Z.ai
34.2% ±1.5Log ↗
11GLM 5.1 Open source
Z.ai
34.0% ±1.5Log ↗
12DeepSeek V4 Flash 0731 Open source
DeepSeek
33.6% ±1.5Log ↗
13GLM 4.7 Open source
Z.ai
32.2% ±1.5Log ↗
14Inkling Small (xhigh) Open source
Thinking Machines
19.1% ±1.2Log ↗
15Gemma 4 31B Open source
Google
10.4% ±1.0Log ↗

Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.