agents benchmark · included in ECI
LMCA
LMCA benchmark (score column: Score).
Results
167
Random baseline
0.0%
Score ceiling
85%
Released
12 Aug 2026
Score by model release date
Best score by organisation
Leaderboard
| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 Anthropic | 68.2% | max | 22 Sep 2026 | — | Source ↗ | |
| 2 | Claude Fable 5.1 (unknown) Anthropic | 65.5% | unknown | 1 Sep 2026 | — | Source ↗ | |
| 3 | Claude Fable 5.1 (medium) Anthropic | 65.3% | medium | 1 Sep 2026 | — | Source ↗ | |
| 4 | Claude Fable 5.1 Anthropic | 65.1% | high | 1 Sep 2026 | — | Source ↗ | |
| 5 | Claude Fable 5.1 (xhigh) Anthropic | 65.1% | xhigh | 1 Sep 2026 | — | Source ↗ | |
| 6 | Claude Opus 5 (xhigh) Anthropic | 64.5% | xhigh | 24 Jul 2026 | — | Source ↗ | |
| 7 | GPT-6 Astra (xhigh) OpenAI | 64.4% | xhigh | 3 Sep 2026 | — | Source ↗ | |
| 8 | GPT-6 Astra (unknown thinking) OpenAI | 64.1% | unknown | 3 Sep 2026 | — | Source ↗ | |
| 9 | GPT-6 Astra OpenAI | 63.8% | high | 3 Sep 2026 | — | Source ↗ | |
| 10 | Claude Opus 5 Anthropic | 63.7% | high | 24 Jul 2026 | — | Source ↗ | |
| 11 | Claude Fable 5.1 (low) Anthropic | 63.5% | low | 1 Sep 2026 | — | Source ↗ | |
| 12 | GPT-6 Astra (medium) OpenAI | 62.9% | medium | 3 Sep 2026 | — | Source ↗ | |
| 13 | Claude Opus 5 (medium) Anthropic | 62.9% | medium | 24 Jul 2026 | — | Source ↗ | |
| 14 | Claude Opus 5 (low) Anthropic | 62.6% | low | 24 Jul 2026 | — | Source ↗ | |
| 15 | GPT-6 Astra (low) OpenAI | 62.2% | low | 3 Sep 2026 | — | Source ↗ | |
| 16 | Claude Fable 5 Anthropic | 61.2% | high | 9 Jun 2026 | — | Source ↗ | |
| 17 | Claude Fable 5 (xhigh) Anthropic | 61.0% | xhigh | 9 Jun 2026 | — | Source ↗ | |
| 18 | Claude Fable 5 (medium) Anthropic | 60.3% | medium | 9 Jun 2026 | — | Source ↗ | |
| 19 | Claude Fable 5 (low) Anthropic | 59.6% | low | 9 Jun 2026 | — | Source ↗ | |
| 20 | GPT-5.6 Sol (pro, max) OpenAI | 59.2% | promax | 9 Jul 2026 | — | Source ↗ | |
| 21 | GPT-6 Sol OpenAI | 59.1% | max | 22 Sep 2026 | — | Source ↗ | |
| 22 | GPT-5.6 Sol (xhigh) OpenAI | 58.5% | xhigh | 9 Jul 2026 | — | Source ↗ | |
| 23 | GPT-5.6 Sol OpenAI | 58.4% | max | 9 Jul 2026 | — | Source ↗ | |
| 24 | Claude Opus 4.8 Anthropic | 57.5% | max | 28 May 2026 | — | Source ↗ | |
| 25 | GPT-5.6 Sol (medium) OpenAI | 56.4% | medium | 9 Jul 2026 | — | Source ↗ | |
| 26 | GPT-5.6 Sol (low) OpenAI | 55.9% | low | 9 Jul 2026 | — | Source ↗ | |
| 27 | Claude Opus 4.6 Anthropic | 55.8% | max | 5 Feb 2026 | — | Source ↗ | |
| 28 | GLM-5.3 (unknown thinking) Z.ai (Zhipu AI) | 55.5% | unknown | 14 Aug 2026 | — | Source ↗ | |
| 29 | GPT-5.6 Terra OpenAI | 54.9% | max | 9 Jul 2026 | — | Source ↗ | |
| 30 | GPT-5.5 (xhigh) OpenAI | 54.3% | xhigh | 23 Apr 2026 | — | Source ↗ | |
| 31 | GPT-5.5 Pro (xhigh) OpenAI | 53.9% | xhigh | 23 Apr 2026 | — | Source ↗ | |
| 32 | Muse Spark 1.3 (unknown) Meta AI | 53.9% | unknown | 2 Sep 2026 | — | Source ↗ | |
| 33 | Gemini 3.1 Pro Preview Google | 53.8% | high | 19 Feb 2026 | — | Source ↗ | |
| 34 | Gemini 3.8 Flash (medium) Google DeepMind | 52.9% | medium | 2 Sep 2026 | — | Source ↗ | |
| 35 | GPT-5.6 Sol (none) OpenAI | 52.8% | none | 9 Jul 2026 | — | Source ↗ | |
| 36 | Kimi K3 Open source MoonshotAI | 52.7% | max | 16 Jul 2026 | — | Source ↗ | |
| 37 | GPT-5.6 Terra (xhigh) OpenAI | 52.2% | xhigh | 9 Jul 2026 | — | Source ↗ | |
| 38 | Claude Opus 4.7 Anthropic | 52.2% | max | 16 Apr 2026 | — | Source ↗ | |
| 39 | GPT-5.4 (xhigh) OpenAI | 52.0% | xhigh | 5 Mar 2026 | — | Source ↗ | |
| 40 | Gemini 3.8 Flash (unknown) Google DeepMind | 51.5% | unknown | 2 Sep 2026 | — | Source ↗ | |
| 41 | Gemini 3.8 Flash (low) Google DeepMind | 51.2% | low | 2 Sep 2026 | — | Source ↗ | |
| 42 | Gemini 3.7 Flash Google | 50.4% | high | 13 Aug 2026 | — | Source ↗ | |
| 43 | Claude Sonnet 5 Anthropic | 50.0% | high | 30 Jun 2026 | — | Source ↗ | |
| 44 | Muse Spark 1.1 Meta | 49.9% | high | 9 Jul 2026 | — | Source ↗ | |
| 45 | Claude Sonnet 5 (medium) Anthropic | 49.7% | medium | 30 Jun 2026 | — | Source ↗ | |
| 46 | Claude Sonnet 5 (low) Anthropic | 49.6% | low | 30 Jun 2026 | — | Source ↗ | |
| 47 | GPT-5.6 Terra (medium) OpenAI | 49.6% | medium | 9 Jul 2026 | — | Source ↗ | |
| 48 | GPT-5.6 Terra (low) OpenAI | 49.3% | low | 9 Jul 2026 | — | Source ↗ | |
| 49 | Gemini 3.7 Flash (medium) Google DeepMind | 49.2% | medium | 13 Aug 2026 | — | Source ↗ | |
| 50 | Claude Sonnet 5 (xhigh) Anthropic | 48.8% | xhigh | 30 Jun 2026 | — | Source ↗ | |
| 51 | GPT-5.6 Luna OpenAI | 48.5% | max | 9 Jul 2026 | — | Source ↗ | |
| 52 | GPT-5.6 Luna (xhigh) OpenAI | 48.5% | xhigh | 9 Jul 2026 | — | Source ↗ | |
| 53 | Grok 4.6 (xhigh) xAI | 48.5% | xhigh | 12 Aug 2026 | — | Source ↗ | |
| 54 | Muse Spark 1.2 (xhigh) Meta AI | 48.4% | xhigh | 5 Aug 2026 | — | Source ↗ | |
| 55 | Gemini 3.5 Flash Google | 47.1% | high | 19 May 2026 | — | Source ↗ | |
| 56 | Gemini 3.7 Flash (low) Google DeepMind | 47.1% | low | 13 Aug 2026 | — | Source ↗ | |
| 57 | DeepSeek V4.1 Flash (unknown) Open source DeepSeek | 47.0% | unknown | 9 Sep 2026 | — | Source ↗ | |
| 58 | GPT-5.6 Terra (none) OpenAI | 46.5% | none | 9 Jul 2026 | — | Source ↗ | |
| 59 | Claude Sonnet 4.6 Anthropic | 46.5% | max | 17 Feb 2026 | — | Source ↗ | |
| 60 | Qwen3.8 Max (xhigh) Alibaba | 46.2% | xhigh | 2 Aug 2026 | — | Source ↗ | |
| 61 | GLM 5.2 Open source Z.ai | 45.8% | max | 16 Jun 2026 | — | Source ↗ | |
| 62 | DeepSeek V4 Pro 0813 (unknown) Open source DeepSeek | 45.5% | unknown | 13 Aug 2026 | — | Source ↗ | |
| 63 | Grok 4.5 SpaceXAI | 45.2% | high | 8 Jul 2026 | — | Source ↗ | |
| 64 | Gemini 3.6 Flash Google | 44.9% | high | 21 Jul 2026 | — | Source ↗ | |
| 65 | GPT-6 Luna OpenAI | 44.5% | max | 22 Sep 2026 | — | Source ↗ | |
| 66 | Claude Opus 4.5 Anthropic | 44.5% | high | 24 Nov 2025 | — | Source ↗ | |
| 67 | GPT-5.6 Luna (medium) OpenAI | 44.3% | medium | 9 Jul 2026 | — | Source ↗ | |
| 68 | Qwen3.7 Max Qwen | 44.0% | — | 19 May 2026 | — | Source ↗ | |
| 69 | GPT-5.2 (xhigh) OpenAI | 43.9% | xhigh | 11 Dec 2025 | — | Source ↗ | |
| 70 | GPT-5.1 OpenAI | 43.9% | high | 13 Nov 2025 | — | Source ↗ | |
| 71 | GPT-5.6 Luna (low) OpenAI | 43.8% | low | 9 Jul 2026 | — | Source ↗ | |
| 72 | Gemini 3 Flash Preview Google | 43.1% | high | 17 Dec 2025 | — | Source ↗ | |
| 73 | Qwen 3.6 Max (Preview) Alibaba | 42.5% | — | 20 Apr 2026 | — | Source ↗ | |
| 74 | DeepSeek V4 Flash 0731 Open source DeepSeek | 41.7% | max | 31 Jul 2026 | — | Source ↗ | |
| 75 | GPT-5.6 Luna (none) OpenAI | 41.4% | none | 9 Jul 2026 | — | Source ↗ | |
| 76 | Qwen3.8 27B Open source Qwen | 41.4% | — | 14 Aug 2026 | — | Source ↗ | |
| 77 | DeepSeek v4 Pro (high) Open source DeepSeek | 41.2% | max | 24 Apr 2026 | — | Source ↗ | |
| 78 | GPT-5.4 mini (xhigh) OpenAI | 40.8% | xhigh | 17 Mar 2026 | — | Source ↗ | |
| 79 | GPT-5 OpenAI | 40.0% | high | 7 Aug 2025 | — | Source ↗ | |
| 80 | o3 OpenAI | 39.7% | high | 16 Apr 2025 | — | Source ↗ | |
| 81 | Gemma 4 31B Open source Google | 39.3% | — | 2 Apr 2026 | — | Source ↗ | |
| 82 | Claude Sonnet 4.5 (unknown thinking) Anthropic | 38.8% | unknown | 29 Sep 2025 | — | Source ↗ | |
| 83 | Grok 4.20 SpaceXAI | 38.7% | — | 17 Feb 2026 | — | Source ↗ | |
| 84 | o3 Pro OpenAI | 38.5% | high | 10 Jun 2025 | — | Source ↗ | |
| 85 | Grok 4.3 Beta xAI | 38.3% | high | 17 Apr 2026 | — | Source ↗ | |
| 86 | Qwen3.5 397B A17B Open source Qwen | 37.9% | — | 13 Feb 2026 | — | Source ↗ | |
| 87 | Gemini 3.5 Flash Lite Google | 37.6% | high | 21 Jul 2026 | — | Source ↗ | |
| 88 | Inkling (xhigh) Open source Thinking Machines | 37.6% | xhigh | 15 Jul 2026 | — | Source ↗ | |
| 89 | Qwen3.7 Plus Qwen | 37.6% | — | 2 Jun 2026 | — | Source ↗ | |
| 90 | Claude Opus 4 Anthropic | 37.4% | unknown | 22 May 2025 | — | Source ↗ | |
| 91 | Kimi K2.6 Open source MoonshotAI | 37.3% | — | 20 Apr 2026 | — | Source ↗ | |
| 92 | Claude Opus 4.1 (unknown thinking) Anthropic | 37.1% | unknown | 5 Aug 2025 | — | Source ↗ | |
| 93 | Nemotron 3 Ultra Open source NVIDIA | 36.9% | — | 4 Jun 2026 | — | Source ↗ | |
| 94 | GPT-5.4 nano (xhigh) OpenAI | 36.9% | xhigh | 17 Mar 2026 | — | Source ↗ | |
| 95 | Qwen3.5 Plus 2026-02-15 Qwen | 36.4% | — | 16 Feb 2026 | — | Source ↗ | |
| 96 | DeepSeek v4 Flash (max) Open source DeepSeek | 35.9% | max | 24 Apr 2026 | — | Source ↗ | |
| 97 | Gemini 3.1 Flash Lite Google | 35.0% | high | 3 Mar 2026 | — | Source ↗ | |
| 98 | Gemini 2.5 Pro (Jun 2025) Google DeepMind | 34.8% | — | 5 Jun 2025 | — | Source ↗ | |
| 99 | Qwen3.6 27B Open source Qwen | 34.5% | — | 22 Apr 2026 | — | Source ↗ | |
| 100 | GPT-5 Mini OpenAI | 34.2% | high | 7 Aug 2025 | — | Source ↗ | |
| 101 | Qwen3.5-27B Open source Qwen | 34.0% | — | 25 Feb 2026 | — | Source ↗ | |
| 102 | MiniMax M3 Open source MiniMax | 33.7% | — | 1 Jun 2026 | — | Source ↗ | |
| 103 | Qwen 3.6 Plus (2026-04-02) Alibaba | 33.1% | — | 31 Mar 2026 | — | Source ↗ | |
| 104 | Qwen3.5-122B-A10B Open source Qwen | 32.2% | none | 25 Feb 2026 | — | Source ↗ | |
| 105 | Qwen 3.6 Flash (2026-04-16) Alibaba | 31.0% | — | 27 Apr 2026 | — | Source ↗ | |
| 106 | Claude Haiku 4.5 (unknown thinking) Anthropic | 30.9% | unknown | 15 Oct 2025 | — | Source ↗ | |
| 107 | Qwen3.6 35B A3B Open source Qwen | 29.7% | — | 27 Apr 2026 | — | Source ↗ | |
| 108 | Gemma 4 26B A4B Open source Google | 29.7% | — | 2 Apr 2026 | — | Source ↗ | |
| 109 | MiMo-V2.5-Pro Open source Xiaomi | 29.5% | — | 22 Apr 2026 | — | Source ↗ | |
| 110 | Qwen3.5-35B-A3B Open source Qwen | 29.5% | — | 24 Feb 2026 | — | Source ↗ | |
| 111 | Qwen3-235B-A22B-Thinking (Jul 2025) Open source Alibaba | 29.3% | — | 25 Jul 2025 | — | Source ↗ | |
| 112 | Qwen3.5-Flash Qwen | 29.1% | — | 25 Feb 2026 | — | Source ↗ | |
| 113 | DeepSeek V3.2 Exp Open source DeepSeek | 29.1% | thinking | 29 Sep 2025 | — | Source ↗ | |
| 114 | Claude Sonnet 4 (unknown thinking) Anthropic | 29.0% | unknown | 22 May 2025 | — | Source ↗ | |
| 115 | DeepSeek-V3.2 (Thinking; Novita) Open source DeepSeek | 28.8% | — | 1 Dec 2025 | — | Source ↗ | |
| 116 | DeepSeek V3.1 Terminus Open source DeepSeek | 28.6% | — | 22 Sep 2025 | — | Source ↗ | |
| 117 | Qwen3-Max-Instruct Alibaba | 28.3% | — | 24 Sep 2025 | — | Source ↗ | |
| 118 | Gemini 2.5 Flash (Jun 2025) Google DeepMind | 27.5% | — | 17 Jun 2025 | — | Source ↗ | |
| 119 | o4 Mini OpenAI | 26.5% | high | 16 Apr 2025 | — | Source ↗ | |
| 120 | Mistral Medium 3.5 Open source Mistral | 26.1% | — | 28 Apr 2026 | — | Source ↗ | |
| 121 | GPT-4.1 OpenAI | 25.6% | — | 14 Apr 2025 | — | Source ↗ | |
| 122 | Qwen3 235B A22B Open source Qwen | 25.0% | — | 28 Apr 2025 | — | Source ↗ | |
| 123 | Qwen3.5-9B Open source Qwen | 24.5% | — | 24 Feb 2026 | — | Source ↗ | |
| 124 | DeepSeek V3.1 Open source DeepSeek | 24.3% | thinking | 21 Aug 2025 | — | Source ↗ | |
| 125 | Qwen Plus (Apr 2025) Alibaba | 24.0% | — | 28 Apr 2025 | — | Source ↗ | |
| 126 | Qwen3-235B-A22B-Instruct (Jul 2025) Open source Alibaba | 23.5% | — | 25 Jul 2025 | — | Source ↗ | |
| 127 | Qwen3-30B-A3B-Instruct (Jul 2025) Open source Alibaba | 22.4% | — | 29 Jul 2025 | — | Source ↗ | |
| 128 | o1 OpenAI | 22.3% | high | 17 Dec 2024 | — | Source ↗ | |
| 129 | gpt-oss-120b Open source OpenAI | 22.1% | high | 5 Aug 2025 | — | Source ↗ | |
| 130 | GPT-4.1 Mini OpenAI | 21.1% | — | 14 Apr 2025 | — | Source ↗ | |
| 131 | Mistral Small 4 Open source Mistral | 20.6% | — | 16 Mar 2026 | — | Source ↗ | |
| 132 | Qwen3-30B-A3B-Thinking (Jul 2025) Open source Alibaba | 19.5% | — | 30 Jul 2025 | — | Source ↗ | |
| 133 | Mistral Medium 3.1 Mistral | 19.1% | — | 13 Aug 2025 | — | Source ↗ | |
| 134 | o3 Mini OpenAI | 19.0% | high | 31 Jan 2025 | — | Source ↗ | |
| 135 | Qwen3 14B Open source Qwen | 18.2% | — | 29 Apr 2025 | — | Source ↗ | |
| 136 | gemini-2.5-flash-lite-preview-06-17 (Default thinking length) Google DeepMind | 18.1% | — | 17 Jun 2025 | — | Source ↗ | |
| 137 | Llama 3.3 70B Open source Meta AI | 17.5% | — | 6 Dec 2024 | — | Source ↗ | |
| 138 | Qwen3 32B Open source Qwen | 17.3% | — | 29 Apr 2025 | — | Source ↗ | |
| 139 | GPT-4 (Jun 2023) OpenAI | 17.1% | — | 13 Jun 2023 | — | Source ↗ | |
| 140 | Claude 3 Opus Anthropic | 17.0% | — | 29 Feb 2024 | — | Source ↗ | |
| 141 | Mistral Large 3 Open source Mistral AI | 16.7% | — | 2 Dec 2025 | — | Source ↗ | |
| 142 | GPT-4o (May 2024) OpenAI | 16.6% | — | 13 May 2024 | — | Source ↗ | |
| 143 | Llama 4 Maverick Open source Meta | 15.9% | — | 6 Apr 2025 | — | Source ↗ | |
| 144 | Qwen3 30B A3B Open source Qwen | 15.8% | — | 29 Apr 2025 | — | Source ↗ | |
| 145 | DeepSeek-V3 (Mar 2025) Open source DeepSeek | 15.5% | — | 24 Mar 2025 | — | Source ↗ | |
| 146 | DeepSeek V3.2 Open source DeepSeek | 15.2% | — | 1 Dec 2025 | — | Source ↗ | |
| 147 | Llama 3.1-70B Open source Meta AI | 14.8% | — | 23 Jul 2024 | — | Source ↗ | |
| 148 | gpt-oss-20b Open source OpenAI | 14.5% | high | 5 Aug 2025 | — | Source ↗ | |
| 149 | Qwen2.5-72B Open source Alibaba | 13.4% | — | 19 Sep 2024 | — | Source ↗ | |
| 150 | Gemma 3 27B Open source Google | 12.3% | — | 12 Mar 2025 | — | Source ↗ | |
| 151 | Llama 4 Scout Open source Meta | 12.0% | — | 5 Apr 2025 | — | Source ↗ | |
| 152 | GPT-4o-mini OpenAI | 10.4% | — | 18 Jul 2024 | — | Source ↗ | |
| 153 | Cohere Command A Open source Cohere | 10.3% | — | 13 Mar 2025 | — | Source ↗ | |
| 154 | GPT-4 Turbo (Nov 2023) OpenAI | 9.8% | — | 25 Jan 2024 | — | Source ↗ | |
| 155 | GPT-3.5 Turbo (Jan 2024) OpenAI | 9.7% | — | 25 Jan 2024 | — | Source ↗ | |
| 156 | c4ai-command-r-08-2024 | 9.2% | — | 30 Aug 2024 | — | Source ↗ | |
| 157 | Claude 3 Haiku Anthropic | 8.8% | — | 7 Mar 2024 | — | Source ↗ | |
| 158 | Qwen3 8B Open source Qwen | 8.8% | — | 28 Apr 2025 | — | Source ↗ | |
| 159 | GPT-5 Nano OpenAI | 7.9% | high | 7 Aug 2025 | — | Source ↗ | |
| 160 | Gemma 2 27B Open source Google | 7.1% | — | 24 Jun 2024 | — | Source ↗ | |
| 161 | Qwen2.5-7B Open source Alibaba | 6.4% | — | 19 Sep 2024 | — | Source ↗ | |
| 162 | Ministral 3B Mistral AI | 5.5% | — | 16 Oct 2024 | — | Source ↗ | |
| 163 | GPT-4.1 Nano OpenAI | 5.5% | — | 14 Apr 2025 | — | Source ↗ | |
| 164 | Llama 3.1-8B Open source Meta AI | 5.4% | — | 23 Jul 2024 | — | Source ↗ | |
| 165 | Command R+ Open source Cohere,Cohere Labs (formerly Cohere for AI) | 5.0% | — | 30 Aug 2024 | — | Source ↗ | |
| 166 | Gemma 3 12B Open source Google | 4.5% | — | 12 Mar 2025 | — | Source ↗ | |
| 167 | Gemma 3 4B Open source Google | 2.8% | — | 12 Mar 2025 | — | Source ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.