math benchmark · included in ECI
MATH level 5
The hardest subset of the MATH competition dataset.
Results
41
Random baseline
0.0%
Score ceiling
100%
Released
5 Mar 2021
Score by model release date
Best score by organisation
Leaderboard
| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-R1 (May 2025) Open source DeepSeek | 96.6% ±0.4 | — | 28 May 2025 | 29 May 2025 | Source ↗ | |
| 2 | R1 Open source DeepSeek | 93.1% ±0.7 | — | 20 Jan 2025 | 31 Jan 2025 | Source ↗ | |
| 3 | DeepSeek-R1-Distill-Llama-70B Open source DeepSeek | 89.9% ±0.7 | — | 20 Jan 2025 | 10 Mar 2025 | Source ↗ | |
| 4 | DeepSeek-R1-Distill-Qwen-14B Open source DeepSeek | 87.1% ±0.7 | — | 20 Jan 2025 | 11 Mar 2025 | Source ↗ | |
| 5 | DeepSeek-V3 (Mar 2025) Open source DeepSeek | 75.5% ±0.9 | — | 24 Mar 2025 | 1 Apr 2025 | Source ↗ | |
| 6 | Gemma 3 27B Open source Google | 74.0% ±1.1 | — | 12 Mar 2025 | 13 Mar 2025 | Source ↗ | |
| 7 | Llama 4 Maverick (FP8) Open source Meta AI | 73.0% ±1.1 | — | 5 Apr 2025 | 8 Apr 2025 | Source ↗ | |
| 8 | Qwen3 235B A22B Open source Qwen | 68.9% ±0.9 | — | 28 Apr 2025 | 3 Jun 2025 | Source ↗ | |
| 9 | Phi 4 Open source Microsoft | 64.9% ±1.1 | — | 12 Dec 2024 | 31 Jan 2025 | Source ↗ | |
| 10 | DeepSeek V3 Open source DeepSeek | 64.9% ±1.0 | — | 26 Dec 2024 | 27 Jan 2025 | Source ↗ | |
| 11 | Qwen2.5-72B Open source Alibaba | 63.2% ±1.1 | — | 19 Sep 2024 | 27 Jan 2025 | Source ↗ | |
| 12 | Llama 4 Scout Open source Meta | 62.3% ±1.2 | — | 5 Apr 2025 | 8 Apr 2025 | Source ↗ | |
| 13 | Qwen2.5-32B Open source Alibaba | 56.1% ±1.1 | — | 17 Sep 2024 | 30 Jan 2025 | Source ↗ | |
| 14 | Mistral Large 2 (Nov 2024) Open source Mistral AI | 50.3% ±1.1 | — | 18 Nov 2024 | 25 Feb 2025 | Source ↗ | |
| 15 | Llama 3.1-405B Open source Meta AI | 49.8% ±1.2 | — | 23 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 16 | Mistral Small 3.1 Open source Mistral AI | 46.8% ±1.2 | — | 17 Mar 2025 | 18 Mar 2025 | Source ↗ | |
| 17 | Mistral Small 3 Open source Mistral | 44.8% ±1.1 | — | 30 Jan 2025 | 30 Jan 2025 | Source ↗ | |
| 18 | Mistral Large 2 (Jul 2024) Open source Mistral AI | 44.8% ±1.1 | — | 24 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 19 | Tulu 3 (Tülu 3) 70B Open source Allen Institute for AI,University of Washington | 42.7% ±1.0 | — | 21 Nov 2024 | 27 Jan 2025 | Source ↗ | |
| 20 | Llama 3.3 70B Open source Meta AI | 41.6% ±1.2 | — | 6 Dec 2024 | 27 Jan 2025 | Source ↗ | |
| 21 | Llama 3.2 90B Open source Meta AI | 39.4% ±1.0 | — | 24 Sep 2024 | 27 Jan 2025 | Source ↗ | |
| 22 | Qwen2-72B Open source Alibaba | 39.1% ±1.0 | — | 7 Jun 2024 | 27 Jan 2025 | Source ↗ | |
| 23 | Llama 3.1-70B Open source Meta AI | 36.7% ±1.0 | — | 23 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 24 | Gemma 2 27B Open source Google | 27.9% ±1.0 | — | 24 Jun 2024 | 27 Jan 2025 | Source ↗ | |
| 25 | WizardLM-2 8x22B Open source microsoft | 25.7% ±0.9 | — | 16 Apr 2024 | 27 Jan 2025 | Source ↗ | |
| 26 | Yi-1.5-34B Open source 01.AI | 25.5% ±0.9 | — | 13 May 2024 | 27 Jan 2025 | Source ↗ | |
| 27 | Mixtral 8x22B Open source Mistral AI | 24.2% ±0.8 | — | 17 Apr 2024 | 27 Jan 2025 | Source ↗ | |
| 28 | Llama 3.1-8B Open source Meta AI | 22.9% ±0.9 | — | 23 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 29 | Hermes 2 Theta Llama-3 70B Open source Nous Research,Arcee AI | 22.7% ±0.8 | — | 20 Jun 2024 | 27 Jan 2025 | Source ↗ | |
| 30 | Llama 3-70B Open source Meta AI | 22.6% ±0.8 | — | 18 Apr 2024 | 27 Jan 2025 | Source ↗ | |
| 31 | Gemma 2 9B Open source Google DeepMind | 21.0% ±0.9 | — | 24 Jun 2024 | 27 Jan 2025 | Source ↗ | |
| 32 | phi-3-medium 14B Open source Microsoft | 17.6% ±0.7 | — | 23 Apr 2024 | 31 Jan 2025 | Source ↗ | |
| 33 | Ministral 8B Open source Mistral AI | 14.9% ±0.7 | — | 16 Oct 2024 | 27 Jan 2025 | Source ↗ | |
| 34 | DBRX Open source Databricks | 11.7% ±0.7 | — | 27 Mar 2024 | 27 Jan 2025 | Source ↗ | |
| 35 | Mistral Nemo Open source Mistral | 10.8% ±0.6 | — | 18 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 36 | Mixtral 8x7B Open source Mistral AI | 10.0% ±0.5 | — | 11 Dec 2023 | 27 Jan 2025 | Source ↗ | |
| 37 | DeepSeek LLM 67B Open source DeepSeek | 6.4% ±0.4 | — | 29 Nov 2023 | 27 Jan 2025 | Source ↗ | |
| 38 | Llama 3-8B Open source Meta AI | 6.1% ±0.4 | — | 18 Apr 2024 | 27 Jan 2025 | Source ↗ | |
| 39 | Yi-34B Open source 01.AI | 5.1% ±0.3 | — | 2 Nov 2023 | 27 Jan 2025 | Source ↗ | |
| 40 | Mistral 7B v0.3 Open source Mistral AI | 3.7% ±0.3 | — | 27 May 2024 | 27 Jan 2025 | Source ↗ | |
| 41 | Llama 2-70B Open source Meta AI | 3.3% ±0.3 | — | 18 Jul 2023 | 27 Jan 2025 | Source ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.