other benchmark · included in ECI
PIQA
PIQA benchmark (score column: Score).
Results
34
Random baseline
50.0%
Score ceiling
100%
Released
26 Nov 2019
Score by model release date
Best score by organisation
Leaderboard
| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | Phi-3.5-MoE Open source Microsoft | 88.6% | — | 17 Aug 2024 | — | Source ↗ | |
| 2 | Llama 3.1-405B Open source Meta AI | 85.9% | — | 23 Jul 2024 | — | Source ↗ | |
| 3 | Falcon-180B Open source Technology Innovation Institute | 84.9% | — | 6 Sep 2023 | — | Source ↗ | |
| 4 | DeepSeek V3 Open source DeepSeek | 84.7% | — | 26 Dec 2024 | — | Source ↗ | |
| 5 | DeepSeek-V2 (MoE-236B, May 2024) Open source DeepSeek | 83.9% | — | 7 May 2024 | — | Source ↗ | |
| 6 | Gemma 2 9B Open source Google DeepMind | 83.7% | — | 24 Jun 2024 | — | Source ↗ | |
| 7 | Mixtral 8x7B Open source Mistral AI | 83.6% | — | 11 Dec 2023 | — | Source ↗ | |
| 8 | Mistral Nemo Open source Mistral | 83.5% | — | 18 Jul 2024 | — | Source ↗ | |
| 9 | Stable Beluga 2 Open source Stability AI | 83.3% | — | 20 Jul 2023 | — | Source ↗ | |
| 10 | Mistral 7B v0.1 Open source Mistral AI | 83.0% | — | 27 Sep 2023 | — | Source ↗ | |
| 11 | Falcon-40B Open source Technology Innovation Institute | 83.0% | — | 25 May 2023 | — | Source ↗ | |
| 12 | Llama 2-70B Open source Meta AI | 82.8% | — | 18 Jul 2023 | — | Source ↗ | |
| 13 | LLaMA-65B Open source Meta AI | 82.8% | — | 24 Feb 2023 | — | Source ↗ | |
| 14 | Qwen2.5-72B Open source Alibaba | 82.6% | — | 19 Sep 2024 | — | Source ↗ | |
| 15 | LLaMA-33B Open source Meta AI | 82.3% | — | 24 Feb 2023 | — | Source ↗ | |
| 16 | Mistral 7B v0.2 Open source Mistral AI | 82.2% | — | 11 Dec 2023 | — | Source ↗ | |
| 17 | MPT-30B Open source MosaicML | 81.9% | — | 22 Jun 2023 | — | Source ↗ | |
| 18 | Llama 3.1-8B Open source Meta AI | 81.2% | — | 23 Jul 2024 | — | Source ↗ | |
| 19 | Gemma 7B Open source Google DeepMind | 81.2% | — | 21 Feb 2024 | — | Source ↗ | |
| 20 | Phi-3.5-mini Open source Microsoft | 81.0% | — | 16 Aug 2024 | — | Source ↗ | |
| 21 | Llama 2-13B Open source Meta AI | 80.8% | — | 18 Jul 2023 | — | Source ↗ | |
| 22 | MPT-7B Open source MosaicML | 80.6% | — | 5 May 2023 | — | Source ↗ | |
| 23 | Falcon-7B Open source Technology Innovation Institute | 80.3% | — | 24 Apr 2023 | — | Source ↗ | |
| 24 | LLaMA-13B Open source Meta AI | 80.1% | — | 24 Feb 2023 | — | Source ↗ | |
| 25 | Qwen-14B Open source Alibaba | 79.9% | — | 24 Sep 2023 | — | Source ↗ | |
| 26 | LLaMA-7B Open source Meta AI | 79.8% | — | 24 Feb 2023 | — | Source ↗ | |
| 27 | Llama 2-7B Open source Meta AI | 78.8% | — | 18 Jul 2023 | — | Source ↗ | |
| 28 | Baichuan2-13B Open source Baichuan | 78.1% | — | 6 Sep 2023 | — | Source ↗ | |
| 29 | Qwen-7B Open source Alibaba | 77.9% | — | 28 Sep 2023 | — | Source ↗ | |
| 30 | Gemma 2B Open source Google DeepMind | 77.3% | — | 21 Feb 2024 | — | Source ↗ | |
| 31 | Baichuan1-7B Open source Baichuan | 76.2% | — | 1 Jun 2023 | — | Source ↗ | |
| 32 | XGen-7B Open source Salesforce | 75.5% | — | 27 Jun 2023 | — | Source ↗ | |
| 33 | Dolly 2.0-12b Open source Databricks | 75.4% | — | 11 Apr 2023 | — | Source ↗ | |
| 34 | Cerebras-GPT-13B Open source Cerebras Systems | 73.5% | — | 20 Mar 2023 | — | Source ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.