Task results
Improving through repeated play (EBR-bench)
Models play the relatively obscure board game Earthborne Rangers repeatedly, taking notes, to see whether they improve.
How to read this table
- Higher rows are rated higher. A range like “1–2” means the gap is within the margin of error.
- The score is the share of tries on a shared set of tasks that the model completed (%).
- A gap of 3 means 3 successes in 100 tries; ± is how far the score moves when re-measured.
- Prices are a guide per million tokens, reading then writing.
- Length is how much text the model can read at once (tokens).
± range of the top model
| Rank | Model | Developer | Success rate (%) | Evaluated | Input $ /1M | Output $ /1M | Context | Supports | Weights | Released | Price and specs |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1–4 | GPT-6 Astra | OpenAI | Rating 76.2 ±16.4 | $10 | $50 | 1.05M | OpenAI Released 2026-09 Price $10 / $50 Length 1.05M | ||||
| 1–5 | Claude Opus 5.5 | Anthropic | Rating 71.4 ±13.8 | $4 | $20 | 1M | Anthropic Released 2026-09 Price $4 / $20 Length 1M | ||||
| 1–6 | Gemini 4 Argon | Rating 64.8 ±9.6 | $2 | $10 | 1.05M | Google Released 2026-09 Price $2 / $10 Length 1.05M | |||||
| 1–8 | Claude Fable 5.1 | Anthropic | Rating 57.1 ±11.8 | $10 | $50 | 1M | Anthropic Released 2026-09 Price $10 / $50 Length 1M | ||||
| 2–9 | GPT-6.1 Sol | OpenAI | Rating 54.3 ±11.3 | $2 | $10 | 1.05M | OpenAI Released 2026-09 Price $2 / $10 Length 1.05M | ||||
| 3–9 | GPT-6 Sol | OpenAI | Rating 53.3 ±9.0 | $2 | $10 | 1.05M | OpenAI Released 2026-09 Price $2 / $10 Length 1.05M | ||||
| 4–10 | Claude Opus 5 | Anthropic | Rating 45.7 ±7.0 | $5 | $25 | 1M | Anthropic Released 2026-07 Price $5 / $25 Length 1M | ||||
| 4–10 | GPT-5.6 Sol | OpenAI | Rating 44.8 ±8.1 | $4 | $20 | 1.05M | OpenAI Released 2026-07 Price $4 / $20 Length 1.05M | ||||
| 5–10 | Claude Haiku 5.5 | Anthropic | Rating 43.8 ±3.5 | $0.1 | $0.5 | 1M | Anthropic Released 2026-10 Price $0.1 / $0.5 Length 1M | ||||
| 7–12 | Claude Fable 5 | Anthropic | Rating 39.5 ±3.9 | $10 | $50 | 1M | Anthropic Released 2026-06 Price $10 / $50 Length 1M | ||||
| 10–13 | GPT-5.5 | OpenAI | Rating 34.3 ±3.9 | $5 | $30 | 1.05M | OpenAI Released 2026-04 Price $5 / $30 Length 1.05M | ||||
| 10–15 | Muse Wonder | Meta | Rating 31.4 ±8.1 | $4 | $20 | 1.05M | Meta Released 2026-10 Price $4 / $20 Length 1.05M | ||||
| 11–14 | Grok 4.6 | xAI | Rating 30.5 ±3.2 | $2 | $6 | 500K | xAI Released 2026-08 Price $2 / $6 Length 500K | ||||
| 12–14 | Claude Opus 4.8 | Anthropic | Rating 28.6 ±2.8 | $5 | $25 | 1M | Anthropic Released 2026-05 Price $5 / $25 Length 1M | ||||
| 14–15 | GPT-5.4 | OpenAI | Rating 25.4 | $2.5 | $15 | 1.05M | OpenAI Released 2026-03 Price $2.5 / $15 Length 1.05M | ||||
| 16 | GPT-5.2 | OpenAI | Rating 23.0 | $1.75 | $14 | 400K | OpenAI Released 2025-12 Price $1.75 / $14 Length 400K | ||||
| 17 | Claude Opus 4.7 | Anthropic | Rating 19.0 | $5 | $25 | 1M | Anthropic Released 2026-04 Price $5 / $25 Length 1M | ||||
| 18–19 | Claude Opus 4.5 | Anthropic | Rating 14.3 | $5 | $25 | 200K | Anthropic Released 2025-11 Price $5 / $25 Length 200K | ||||
| 18–19 | Gemini 3.1 Pro (preview) | Rating 14.3 | $2 | $12 | 1.05M | Google Released 2026-02 Price $2 / $12 Length 1.05M | |||||
| 20–21 | Claude Opus 4.6 | Anthropic | Rating 12.7 | $5 | $25 | 1M | Anthropic Released 2026-02 Price $5 / $25 Length 1M | ||||
| 20–21 | GPT-5 | OpenAI | Rating 12.7 | $1.25 | $10 | 400K | OpenAI Released 2025-08 Price $1.25 / $10 Length 400K | ||||
| 22–23 | GLM-5.2 | Z.ai | Rating 9.5 | $1.4 | $4.4 | 1M | Z.ai Released 2026-06 Price $1.4 / $4.4 Length 1M | ||||
| 22–23 | Qwen3.7 Max | Alibaba | Rating 9.5 | $2.5 | $7.5 | 1M | Alibaba Released 2026-05 Price $2.5 / $7.5 Length 1M | ||||
| 24 | Claude Opus 4.1 | Anthropic | Rating 7.9 | — | — | 200K | Anthropic Released 2025-08 Price — Length 200K | ||||
| 25 | Gemini 3.5 Flash | Rating 4.8 | $1.5 | $9 | 1.05M | Google Released 2026-05 Price $1.5 / $9 Length 1.05M | |||||
| 26–27 | Claude Sonnet 4.5 | Anthropic | Rating 2.4 | $3 | $15 | 200K | Anthropic Released 2025-09 Price $3 / $15 Length 200K | ||||
| 26–27 | Kimi K2.6 | Moonshot | Rating 2.4 | $0.95 | $4 | 262.1K | Moonshot Released 2026-04 Price $0.95 / $4 Length 262.1K |
Best success rate over time
2025-08-05Release date2026-10-12
How to read this page
- The better of the scored playthroughs after the learning playthroughs is used, and the value is the share of objectives achieved.
- The success rate is the share of tries that succeeded. A gap of 1 point is 1 success out of 100 tries.
- In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
- Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
- Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.
Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-10-24, data sources