reasoning benchmark · included in ECI
SimpleBench
Trick questions on spatio-temporal and social reasoning where humans outperform models.
Results
26
Random baseline
16.7%
Score ceiling
100%
Released
31 Oct 2024
Score by model release date
Best score by organisation
Leaderboard
| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek V4 Flash 0731 (unknown) Open source DeepSeek | 61.1% | unknown | 31 Jul 2026 | — | Source ↗ | |
| 2 | Kimi K3 Open source MoonshotAI | 60.7% | max | 16 Jul 2026 | — | Source ↗ | |
| 3 | GLM 5.2 Open source Z.ai | 58.8% | unknown | 16 Jun 2026 | — | Source ↗ | |
| 4 | Kimi K2.7 Code Open source MoonshotAI | 57.9% | — | 12 Jun 2026 | — | Source ↗ | |
| 5 | GLM 5.1 Open source Z.ai | 55.1% | — | 7 Apr 2026 | — | Source ↗ | |
| 6 | GLM 5 Open source Z.ai | 53.2% | — | 11 Feb 2026 | — | Source ↗ | |
| 7 | DeepSeek-V3.2-Speciale Open source DeepSeek | 52.6% | — | 1 Dec 2025 | — | Source ↗ | |
| 8 | Inkling Open source Thinking Machines | 50.0% | — | 15 Jul 2026 | — | Source ↗ | |
| 9 | GLM-4.7 (Novita) Open source Z.ai (Zhipu AI) | 47.7% | — | 22 Dec 2025 | — | Source ↗ | |
| 10 | GLM 4.7 Open source Z.ai | 47.7% | — | 22 Dec 2025 | — | Source ↗ | |
| 11 | Kimi K2.5 Open source MoonshotAI | 46.8% | — | 27 Jan 2026 | — | Source ↗ | |
| 12 | DeepSeek v4 (unknown) Open source DeepSeek | 46.3% | unknown | 24 Apr 2026 | — | Source ↗ | |
| 13 | MiniMax M3 Open source MiniMax | 45.8% | — | 1 Jun 2026 | — | Source ↗ | |
| 14 | DeepSeek-R1 (May 2025) Open source DeepSeek | 40.8% | — | 28 May 2025 | — | Source ↗ | |
| 15 | DeepSeek V3.1 Open source DeepSeek | 40.0% | — | 21 Aug 2025 | — | Source ↗ | |
| 16 | Qwen3 235B A22B Open source Qwen | 31.0% | — | 28 Apr 2025 | — | Source ↗ | |
| 17 | R1 Open source DeepSeek | 30.9% | — | 20 Jan 2025 | — | Source ↗ | |
| 18 | Llama 4 Maverick Open source Meta | 27.7% | — | 6 Apr 2025 | — | Source ↗ | |
| 19 | DeepSeek-V3 (Mar 2025) Open source DeepSeek | 27.2% | — | 24 Mar 2025 | — | Source ↗ | |
| 20 | Kimi K2 Instruct Open source Moonshot | 26.3% | — | 12 Jul 2025 | — | Source ↗ | |
| 21 | Llama 3.1-405B Open source Meta AI | 23.0% | — | 23 Jul 2024 | — | Source ↗ | |
| 22 | Mistral Large 2 (Jul 2024) Open source Mistral AI | 22.5% | — | 24 Jul 2024 | — | Source ↗ | |
| 23 | gpt-oss-120b Open source OpenAI | 22.1% | — | 5 Aug 2025 | — | Source ↗ | |
| 24 | Llama 3.3 70B Open source Meta AI | 19.9% | — | 6 Dec 2024 | — | Source ↗ | |
| 25 | DeepSeek V3 Open source DeepSeek | 18.9% | — | 26 Dec 2024 | — | Source ↗ | |
| 26 | Command R+ Open source Cohere,Cohere Labs (formerly Cohere for AI) | 17.4% | — | 30 Aug 2024 | — | Source ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.