math benchmark · included in ECI
MATH level 5
The hardest subset of the MATH competition dataset.
Results
104
Random baseline
0.0%
Score ceiling
100%
Released
5 Mar 2021
Score by model release date
Best score by organisation
Leaderboard
| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | GPT-5 OpenAI | 98.1% ±0.3 | high | 7 Aug 2025 | 29 Oct 2025 | Source ↗ | |
| 2 | GPT-5 (medium) OpenAI | 97.9% ±0.3 | medium | 7 Aug 2025 | 20 Aug 2025 | Source ↗ | |
| 3 | GPT-5 Mini OpenAI | 97.8% ±0.3 | high | 7 Aug 2025 | 30 Oct 2025 | Source ↗ | |
| 4 | o4 Mini OpenAI | 97.8% ±0.3 | high | 16 Apr 2025 | 16 Apr 2025 | Source ↗ | |
| 5 | o3 OpenAI | 97.8% ±0.3 | high | 16 Apr 2025 | 16 Apr 2025 | Source ↗ | |
| 6 | Claude Sonnet 4.5 (32k thinking) Anthropic | 97.7% ±0.4 | 32K | 29 Sep 2025 | 21 Oct 2025 | Source ↗ | |
| 7 | Qwen3-Max-Instruct Alibaba | 97.1% ±0.4 | — | 24 Sep 2025 | 9 Oct 2025 | Source ↗ | |
| 8 | GPT-5 mini (medium) OpenAI | 96.8% ±0.4 | medium | 7 Aug 2025 | 20 Aug 2025 | Source ↗ | |
| 9 | DeepSeek-R1 (May 2025) Open source DeepSeek | 96.6% ±0.4 | — | 28 May 2025 | 29 May 2025 | Source ↗ | |
| 10 | o3 Mini OpenAI | 96.5% ±0.4 | high | 31 Jan 2025 | 13 Feb 2025 | Source ↗ | |
| 11 | Claude Haiku 4.5 Anthropic | 96.4% ±0.5 | 32K | 15 Oct 2025 | 22 Oct 2025 | Source ↗ | |
| 12 | Gemini 2.5 Pro Preview (Jun 2025) Google DeepMind | 95.9% ±0.4 | — | 5 Jun 2025 | 8 May 2025 | Source ↗ | |
| 13 | Gemini 2.5 Pro Preview (Mar 2025) Google DeepMind | 95.6% ±0.4 | — | 31 Mar 2025 | 7 May 2025 | Source ↗ | |
| 14 | GPT-5 nano (medium) OpenAI | 95.2% ±0.5 | medium | 7 Aug 2025 | 20 Aug 2025 | Source ↗ | |
| 15 | o3-mini (medium) OpenAI | 95.2% ±0.5 | medium | 31 Jan 2025 | 31 Jan 2025 | Source ↗ | |
| 16 | GPT-5 Nano OpenAI | 94.9% ±0.5 | high | 7 Aug 2025 | 20 Aug 2025 | Source ↗ | |
| 17 | o1 OpenAI | 94.7% ±0.6 | high | 17 Dec 2024 | 13 Feb 2025 | Source ↗ | |
| 18 | o1 (medium) OpenAI | 94.4% ±0.6 | medium | 17 Dec 2024 | 27 Jan 2025 | Source ↗ | |
| 19 | R1 Open source DeepSeek | 93.1% ±0.7 | — | 20 Jan 2025 | 31 Jan 2025 | Source ↗ | |
| 20 | Claude 3.7 Sonnet (64k thinking) Anthropic | 91.2% ±0.8 | 64K | 24 Feb 2025 | 13 Mar 2025 | Source ↗ | |
| 21 | Grok-3 mini xAI | 90.9% ±0.8 | low | 24 Jun 2025 | 10 Apr 2025 | Source ↗ | |
| 22 | Claude 3.7 Sonnet (32k thinking) Anthropic | 90.0% ±0.8 | 32K | 24 Feb 2025 | 12 Mar 2025 | Source ↗ | |
| 23 | DeepSeek-R1-Distill-Llama-70B Open source DeepSeek | 89.9% ±0.7 | — | 20 Jan 2025 | 10 Mar 2025 | Source ↗ | |
| 24 | o1-mini OpenAI | 89.2% ±0.7 | high | 12 Sep 2024 | 13 Feb 2025 | Source ↗ | |
| 25 | Grok 3 xAI | 88.7% ±0.9 | — | 9 Apr 2025 | 10 Apr 2025 | Source ↗ | |
| 26 | GPT-4.1 Mini OpenAI | 87.3% ±0.7 | — | 14 Apr 2025 | 14 Apr 2025 | Source ↗ | |
| 27 | DeepSeek-R1-Distill-Qwen-14B Open source DeepSeek | 87.1% ±0.7 | — | 20 Jan 2025 | 11 Mar 2025 | Source ↗ | |
| 28 | Claude 3.7 Sonnet (16k thinking) Anthropic | 86.3% ±0.9 | 16K | 24 Feb 2025 | 26 Feb 2025 | Source ↗ | |
| 29 | Claude Opus 4 Anthropic | 85.0% ±1.0 | — | 22 May 2025 | 22 May 2025 | Source ↗ | |
| 30 | Claude Sonnet 4 Anthropic | 84.4% ±1.0 | — | 22 May 2025 | 22 May 2025 | Source ↗ | |
| 31 | o1-mini (medium) OpenAI | 84.3% ±0.8 | medium | 12 Sep 2024 | 27 Jan 2025 | Source ↗ | |
| 32 | Gemini 2.0 Pro Exp (Feb 2025) Google DeepMind | 83.5% ±1.0 | — | 5 Feb 2025 | 6 Feb 2025 | Source ↗ | |
| 33 | GPT-4.1 OpenAI | 83.0% ±0.8 | — | 14 Apr 2025 | 14 Apr 2025 | Source ↗ | |
| 34 | Gemini 2.0 Flash (Feb 2025) Google DeepMind,Google | 82.2% ±0.9 | — | 5 Feb 2025 | 6 Feb 2025 | Source ↗ | |
| 35 | o1-preview OpenAI | 81.6% ±1.1 | — | 12 Sep 2024 | 27 Jan 2025 | Source ↗ | |
| 36 | Mistral Medium 3 Mistral | 81.6% ±0.9 | — | 7 May 2025 | 7 May 2025 | Source ↗ | |
| 37 | GPT-4.5 Preview (Feb 2025) OpenAI | 78.6% ±1.1 | — | 27 Feb 2025 | 28 Feb 2025 | Source ↗ | |
| 38 | DeepSeek-V3 (Mar 2025) Open source DeepSeek | 75.5% ±0.9 | — | 24 Mar 2025 | 1 Apr 2025 | Source ↗ | |
| 39 | Gemma 3 27B Open source Google | 74.0% ±1.1 | — | 12 Mar 2025 | 13 Mar 2025 | Source ↗ | |
| 40 | Llama 4 Maverick (FP8) Open source Meta AI | 73.0% ±1.1 | — | 5 Apr 2025 | 8 Apr 2025 | Source ↗ | |
| 41 | Gemini 1.5 Pro (Sept 2024) Google DeepMind | 70.4% ±1.0 | — | 24 Sep 2024 | 27 Jan 2025 | Source ↗ | |
| 42 | GPT-4.1 Nano OpenAI | 70.0% ±1.0 | — | 14 Apr 2025 | 14 Apr 2025 | Source ↗ | |
| 43 | Qwen3 235B A22B Open source Qwen | 68.9% ±0.9 | — | 28 Apr 2025 | 3 Jun 2025 | Source ↗ | |
| 44 | Claude 3.7 Sonnet Anthropic | 68.2% ±1.1 | — | 24 Feb 2025 | 24 Feb 2025 | Source ↗ | |
| 45 | Qwen2.5-Max Alibaba | 67.2% ±1.1 | — | 25 Jan 2025 | 1 Apr 2025 | Source ↗ | |
| 46 | Qwen Plus (Jan 2025) Alibaba | 65.3% ±1.1 | — | 25 Jan 2025 | 7 Apr 2025 | Source ↗ | |
| 47 | Phi 4 Open source Microsoft | 64.9% ±1.1 | — | 12 Dec 2024 | 31 Jan 2025 | Source ↗ | |
| 48 | DeepSeek V3 Open source DeepSeek | 64.9% ±1.0 | — | 26 Dec 2024 | 27 Jan 2025 | Source ↗ | |
| 49 | Grok-2 (Dec 2024) xAI | 63.5% ±1.3 | — | 12 Dec 2024 | 17 Feb 2025 | Source ↗ | |
| 50 | Qwen2.5-72B Open source Alibaba | 63.2% ±1.1 | — | 19 Sep 2024 | 27 Jan 2025 | Source ↗ | |
| 51 | Llama 4 Scout Open source Meta | 62.3% ±1.2 | — | 5 Apr 2025 | 8 Apr 2025 | Source ↗ | |
| 52 | Gemini 1.5 Flash (Sep 2024) Google DeepMind | 61.9% ±1.1 | — | 24 Sep 2024 | 27 Jan 2025 | Source ↗ | |
| 53 | Claude 3.5 Sonnet (Oct 2024) Anthropic | 56.9% ±1.1 | — | 22 Oct 2024 | 27 Jan 2025 | Source ↗ | |
| 54 | Qwen-Turbo Alibaba | 56.2% ±1.1 | — | 1 Nov 2024 | 7 Apr 2025 | Source ↗ | |
| 55 | Qwen2.5-32B Open source Alibaba | 56.1% ±1.1 | — | 17 Sep 2024 | 30 Jan 2025 | Source ↗ | |
| 56 | GPT-4o (Aug 2024) OpenAI | 53.3% ±1.1 | — | 6 Aug 2024 | 27 Jan 2025 | Source ↗ | |
| 57 | GPT-4o-mini OpenAI | 52.6% ±1.1 | — | 18 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 58 | Claude 3.5 Sonnet (Jun 2024) Anthropic | 51.7% ±1.2 | — | 20 Jun 2024 | 27 Jan 2025 | Source ↗ | |
| 59 | GPT-4o (May 2024) OpenAI | 51.0% ±1.1 | — | 13 May 2024 | 27 Jan 2025 | Source ↗ | |
| 60 | Mistral Large 2 (Nov 2024) Open source Mistral AI | 50.3% ±1.1 | — | 18 Nov 2024 | 25 Feb 2025 | Source ↗ | |
| 61 | GPT-4o (Nov 2024) OpenAI | 49.8% ±1.1 | — | 20 Nov 2024 | 5 Feb 2025 | Source ↗ | |
| 62 | Llama 3.1-405B Open source Meta AI | 49.8% ±1.2 | — | 23 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 63 | Mistral Small 3.1 Open source Mistral AI | 46.8% ±1.2 | — | 17 Mar 2025 | 18 Mar 2025 | Source ↗ | |
| 64 | GPT-4 Turbo (Apr 2024) OpenAI | 46.7% ±1.1 | — | 9 Apr 2024 | 27 Feb 2025 | Source ↗ | |
| 65 | Claude 3.5 Haiku (Oct 2024) Anthropic | 46.4% ±1.2 | — | 22 Oct 2024 | 12 Mar 2025 | Source ↗ | |
| 66 | Mistral Small 3 Open source Mistral | 44.8% ±1.1 | — | 30 Jan 2025 | 30 Jan 2025 | Source ↗ | |
| 67 | Mistral Large 2 (Jul 2024) Open source Mistral AI | 44.8% ±1.1 | — | 24 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 68 | Tulu 3 (Tülu 3) 70B Open source Allen Institute for AI,University of Washington | 42.7% ±1.0 | — | 21 Nov 2024 | 27 Jan 2025 | Source ↗ | |
| 69 | Llama 3.3 70B Open source Meta AI | 41.6% ±1.2 | — | 6 Dec 2024 | 27 Jan 2025 | Source ↗ | |
| 70 | Gemini 1.5 Pro (May 2024) Google DeepMind | 40.7% ±1.1 | — | 14 May 2024 | 27 Jan 2025 | Source ↗ | |
| 71 | GPT-4 Turbo Preview (Nov 2023) OpenAI | 40.0% ±1.1 | — | 6 Nov 2023 | 27 Jan 2025 | Source ↗ | |
| 72 | Llama 3.2 90B Open source Meta AI | 39.4% ±1.0 | — | 24 Sep 2024 | 27 Jan 2025 | Source ↗ | |
| 73 | Qwen2-72B Open source Alibaba | 39.1% ±1.0 | — | 7 Jun 2024 | 27 Jan 2025 | Source ↗ | |
| 74 | Claude 3 Opus Anthropic | 37.5% ±1.1 | — | 29 Feb 2024 | 27 Jan 2025 | Source ↗ | |
| 75 | Llama 3.1-70B Open source Meta AI | 36.7% ±1.0 | — | 23 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 76 | GPT-4 Turbo Preview (January 2024) OpenAI | 35.4% ±1.1 | — | 25 Jan 2024 | 27 Jan 2025 | Source ↗ | |
| 77 | Gemma 2 27B Open source Google | 27.9% ±1.0 | — | 24 Jun 2024 | 27 Jan 2025 | Source ↗ | |
| 78 | WizardLM-2 8x22B Open source microsoft | 25.7% ±0.9 | — | 16 Apr 2024 | 27 Jan 2025 | Source ↗ | |
| 79 | Yi-1.5-34B Open source 01.AI | 25.5% ±0.9 | — | 13 May 2024 | 27 Jan 2025 | Source ↗ | |
| 80 | Gemini 1.5 Flash (May 2024) Google DeepMind | 25.1% ±0.9 | — | 23 May 2024 | 27 Jan 2025 | Source ↗ | |
| 81 | Mistral Large mistralai | 24.5% ±0.9 | — | 26 Feb 2024 | 27 Jan 2025 | Source ↗ | |
| 82 | Mixtral 8x22B Open source Mistral AI | 24.2% ±0.8 | — | 17 Apr 2024 | 27 Jan 2025 | Source ↗ | |
| 83 | GPT-4 (Jun 2023) OpenAI | 23.0% ±0.8 | — | 13 Jun 2023 | 27 Jan 2025 | Source ↗ | |
| 84 | Llama 3.1-8B Open source Meta AI | 22.9% ±0.9 | — | 23 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 85 | Hermes 2 Theta Llama-3 70B Open source Nous Research,Arcee AI | 22.7% ±0.8 | — | 20 Jun 2024 | 27 Jan 2025 | Source ↗ | |
| 86 | Llama 3-70B Open source Meta AI | 22.6% ±0.8 | — | 18 Apr 2024 | 27 Jan 2025 | Source ↗ | |
| 87 | Gemma 2 9B Open source Google DeepMind | 21.0% ±0.9 | — | 24 Jun 2024 | 27 Jan 2025 | Source ↗ | |
| 88 | Claude 3 Sonnet Anthropic | 18.2% ±0.8 | — | 29 Feb 2024 | 27 Jan 2025 | Source ↗ | |
| 89 | phi-3-medium 14B Open source Microsoft | 17.6% ±0.7 | — | 23 Apr 2024 | 31 Jan 2025 | Source ↗ | |
| 90 | GPT-3.5 Turbo (Nov 2023) OpenAI | 15.9% ±0.7 | — | 6 Nov 2023 | 27 Jan 2025 | Source ↗ | |
| 91 | Ministral 8B Open source Mistral AI | 14.9% ±0.7 | — | 16 Oct 2024 | 27 Jan 2025 | Source ↗ | |
| 92 | Claude 3 Haiku Anthropic | 14.9% ±0.7 | — | 7 Mar 2024 | 27 Jan 2025 | Source ↗ | |
| 93 | Ministral 3B Mistral AI | 14.4% ±0.7 | — | 16 Oct 2024 | 27 Jan 2025 | Source ↗ | |
| 94 | Claude 2 Anthropic | 11.7% ±0.7 | — | 11 Jul 2023 | 27 Jan 2025 | Source ↗ | |
| 95 | DBRX Open source Databricks | 11.7% ±0.7 | — | 27 Mar 2024 | 27 Jan 2025 | Source ↗ | |
| 96 | GPT-3.5 Turbo (Jan 2024) OpenAI | 11.6% ±0.6 | — | 25 Jan 2024 | 27 Jan 2025 | Source ↗ | |
| 97 | Gemini 1.0 Pro Google DeepMind | 11.2% ±0.6 | — | 13 Dec 2023 | 27 Jan 2025 | Source ↗ | |
| 98 | Mistral Nemo Open source Mistral | 10.8% ±0.6 | — | 18 Jul 2024 | 27 Jan 2025 | Source ↗ | |
| 99 | Mixtral 8x7B Open source Mistral AI | 10.0% ±0.5 | — | 11 Dec 2023 | 27 Jan 2025 | Source ↗ | |
| 100 | DeepSeek LLM 67B Open source DeepSeek | 6.4% ±0.4 | — | 29 Nov 2023 | 27 Jan 2025 | Source ↗ | |
| 101 | Llama 3-8B Open source Meta AI | 6.1% ±0.4 | — | 18 Apr 2024 | 27 Jan 2025 | Source ↗ | |
| 102 | Yi-34B Open source 01.AI | 5.1% ±0.3 | — | 2 Nov 2023 | 27 Jan 2025 | Source ↗ | |
| 103 | Mistral 7B v0.3 Open source Mistral AI | 3.7% ±0.3 | — | 27 May 2024 | 27 Jan 2025 | Source ↗ | |
| 104 | Llama 2-70B Open source Meta AI | 3.3% ±0.3 | — | 18 Jul 2023 | 27 Jan 2025 | Source ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.