coding benchmark · included in ECI
Cybench
Cybench benchmark (score column: Unguided % Solved).
Results
3
Random baseline
0.0%
Score ceiling
100%
Released
15 Aug 2024
Score by model release date
Best score by organisation
Leaderboard
| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | Llama 3.1-405B Open source Meta AI | 7.5% | — | 23 Jul 2024 | — | Cybench leaderboard ↗ | |
| 2 | Mixtral 8x22B Open source Mistral AI | 7.5% | — | 17 Apr 2024 | — | Cybench leaderboard ↗ | |
| 3 | Llama 3-70B Open source Meta AI | 5.0% | — | 18 Apr 2024 | — | Cybench leaderboard ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.