games benchmark · included in ECI
Balrog
Balrog benchmark (score column: Average progress).
Results
39
Random baseline
0.0%
Score ceiling
100%
Released
20 Nov 2024
Score by model release date
Best score by organisation
Leaderboard
| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra OpenAI | 68.3% | max | 3 Sep 2026 | — | Source ↗ | |
| 2 | Claude Opus 5 Anthropic | 63.4% | max | 24 Jul 2026 | — | Source ↗ | |
| 3 | GPT-5.6 Sol OpenAI | 60.0% | max | 9 Jul 2026 | — | Source ↗ | |
| 4 | Gemini 3 Pro Preview Google DeepMind | 58.1% | — | 18 Nov 2025 | — | Source ↗ | |
| 5 | Gemini 3.1 Pro Preview Google | 57.0% | — | 19 Feb 2026 | — | Source ↗ | |
| 6 | GPT-5.6 Terra OpenAI | 53.2% | max | 9 Jul 2026 | — | Source ↗ | |
| 7 | Gemini 3 Flash Preview Google | 48.1% | — | 17 Dec 2025 | — | Source ↗ | |
| 8 | GPT-5.6 Luna OpenAI | 45.6% | max | 9 Jul 2026 | — | Source ↗ | |
| 9 | Grok 4 xAI | 43.6% | — | 9 Jul 2025 | — | Source ↗ | |
| 10 | Claude Opus 4.5 (no thinking) Anthropic | 43.5% | — | 24 Nov 2025 | — | Source ↗ | |
| 11 | Gemini 2.5 Pro Exp (Mar 2025) Google DeepMind | 43.3% | — | 25 Mar 2025 | — | Source ↗ | |
| 12 | Claude Opus 4.5 (64k thinking) Anthropic | 43.0% | 64K | 24 Nov 2025 | — | Source ↗ | |
| 13 | Claude Opus 4.5 (unknown thinking) Anthropic | 43.0% | unknown | 24 Nov 2025 | — | Source ↗ | |
| 14 | R1 Open source DeepSeek | 34.9% | — | 20 Jan 2025 | — | Source ↗ | |
| 15 | Gemini 2.5 Flash (Jun 2025) Google DeepMind | 33.5% | — | 17 Jun 2025 | — | Source ↗ | |
| 16 | GPT-5 (minimal) OpenAI | 32.8% | minimal | 7 Aug 2025 | — | Source ↗ | |
| 17 | Claude 3.5 Sonnet (Oct 2024) Anthropic | 32.6% | — | 22 Oct 2024 | — | Source ↗ | |
| 18 | GPT-4o (May 2024) OpenAI | 32.3% | — | 13 May 2024 | — | Source ↗ | |
| 19 | Claude Haiku 4.5 Anthropic | 31.2% | — | 15 Oct 2025 | — | Source ↗ | |
| 20 | Grok 3 xAI | 29.5% | — | 9 Apr 2025 | — | Source ↗ | |
| 21 | Reka Flash 3 Open source rekaai | 29.2% | — | 12 Mar 2025 | — | Source ↗ | |
| 22 | Llama 3.1-70B Open source Meta AI | 27.9% | — | 23 Jul 2024 | — | Source ↗ | |
| 23 | Llama 3.2 90B Open source Meta AI | 27.3% | — | 24 Sep 2024 | — | Source ↗ | |
| 24 | Llama 3.3 70B Open source Meta AI | 23.0% | — | 6 Dec 2024 | — | Source ↗ | |
| 25 | Gemini 1.5 Pro (Sept 2024) Google DeepMind | 21.0% | — | 24 Sep 2024 | — | Source ↗ | |
| 26 | DeepSeek-R1-Distill-Qwen-32B Open source DeepSeek | 19.5% | — | 20 Jan 2025 | — | Source ↗ | |
| 27 | Claude 3.5 Haiku (Oct 2024) Anthropic | 19.3% | — | 22 Oct 2024 | — | Source ↗ | |
| 28 | Mistral Nemo Open source Mistral | 17.6% | — | 18 Jul 2024 | — | Source ↗ | |
| 29 | GPT-4o-mini OpenAI | 17.4% | — | 18 Jul 2024 | — | Source ↗ | |
| 30 | Llama 3.2 11B Open source Meta AI | 16.8% | — | 24 Sep 2024 | — | Source ↗ | |
| 31 | Qwen2.5-72B Open source Alibaba | 16.2% | — | 19 Sep 2024 | — | Source ↗ | |
| 32 | Llama 3.1-8B Open source Meta AI | 15.1% | — | 23 Jul 2024 | — | Source ↗ | |
| 33 | Gemini 1.5 Flash (Sep 2024) Google DeepMind | 14.6% | — | 24 Sep 2024 | — | Source ↗ | |
| 34 | Qwen2-VL-72B-Instruct | 12.8% | — | 29 Aug 2024 | — | Source ↗ | |
| 35 | Phi 4 Open source Microsoft | 11.6% | — | 12 Dec 2024 | — | Source ↗ | |
| 36 | Llama 3.2 3B Open source Meta AI | 10.1% | — | 24 Sep 2024 | — | Source ↗ | |
| 37 | Qwen2.5-7B Open source Alibaba | 7.8% | — | 19 Sep 2024 | — | Source ↗ | |
| 38 | Llama 3.2 1B Open source Meta AI | 6.6% | — | 24 Sep 2024 | — | Source ↗ | |
| 39 | Qwen2-VL-7B-Instruct | 3.7% | — | 29 Aug 2024 | — | Source ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.