Reported by others
Editing code in six programming languages (Aider polyglot)
Hard practice problems in C++, Go, Java, JavaScript, Python and Rust that the model solves by editing existing files.
How to read this table
- Higher rows are rated higher. A range like “1–2” means the gap is within the margin of error.
- The score is the value published by the task's authors and other organizations.
- Conditions can differ between models, so small gaps do not mean much.
- Prices are a guide per million tokens, reading then writing.
- Length is how much text the model can read at once (tokens).
| Rank | Model | Developer | Success rate (%) | Input $ /1M | Output $ /1M | Context | Supports | Weights | Released | Price and specs |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-5 | OpenAI | Rating 88.0 | $1.25 | $10 | 400K | OpenAI Released 2025-08 Price $1.25 / $10 Length 400K | |||
| 2 | o3-pro | OpenAI | Rating 84.9 | — | — | 200K | OpenAI Released 2025-06 Price — Length 200K | |||
| 3 | Gemini 2.5 Pro (preview, 2025-05) | Rating 83.1 | — | — | 1.05M | Google Released 2025-05 Price — Length 1.05M | ||||
| 4 | o3 | OpenAI | Rating 81.3 | $2 | $8 | 200K | OpenAI Released 2025-04 Price $2 / $8 Length 200K | |||
| 5 | Grok 4 | xAI | Rating 79.6 | — | — | 256K | xAI Released 2025-07 Price — Length 256K | |||
| 6–7 | DeepSeek-V3.2 (exp) | DeepSeek | Rating 74.2 | — | — | 163.8K | DeepSeek Released 2025-09 Price — Length 163.8K | |||
| 6–7 | DeepSeek-V3.2 | DeepSeek | Rating 74.2 | — | — | 128K | DeepSeek Released 2025-12 Price — Length 128K | |||
| 8–9 | Gemini 2.5 Pro (exp) | Rating 72.9 | — | — | 1.05M | Google Released 2025-03 Price — Length 1.05M | ||||
| 8–9 | Gemini 2.5 Pro (preview, 2025-03) | Rating 72.9 | — | — | 1.05M | Google Released 2025-03 Price — Length 1.05M | ||||
| 10–11 | Claude Opus 4 | Anthropic | Rating 72.0 | — | — | 200K | Anthropic Released 2025-05 Price — Length 200K | |||
| 10–11 | o4-mini | OpenAI | Rating 72.0 | $1.1 | $4.4 | 200K | OpenAI Released 2025-04 Price $1.1 / $4.4 Length 200K | |||
| 12 | DeepSeek-R1 (2025-05) | DeepSeek | Rating 71.4 | — | — | 131.1K | DeepSeek Released 2025-05 Price — Length 131.1K | |||
| 13 | DeepSeek Chat (API) | DeepSeek | Rating 70.2 | — | — | 1M | DeepSeek Released 2025-12 Price — Length 1M | |||
| 14 | Claude 3.7 Sonnet | Anthropic | Rating 64.9 | — | — | 200K | Anthropic Released 2025-02 Price — Length 200K | |||
| 15 | o1 | OpenAI | Rating 61.7 | $15 | $60 | 128K | OpenAI Released 2024-12 Price $15 / $60 Length 128K | |||
| 16 | Claude Sonnet 4 | Anthropic | Rating 61.3 | — | — | 200K | Anthropic Released 2025-05 Price — Length 200K | |||
| 17 | o3-mini | OpenAI | Rating 60.4 | $1.1 | $4.4 | 200K | OpenAI Released 2025-01 Price $1.1 / $4.4 Length 200K | |||
| 18–19 | Qwen3 235B A22B | Alibaba | Rating 59.6 | $0.7 | $2.8 | 128K | Alibaba Released 2025-04 Price $0.7 / $2.8 Length 128K | |||
| 18–19 | Qwen3 235B A22B (instruct) | Alibaba | Rating 59.6 | — | — | 128K | Alibaba Released 2025-07 Price — Length 128K | |||
| 20–21 | Kimi K2 (instruct) | Moonshot | Rating 59.1 | — | — | 131.1K | Moonshot Released 2025-07 Price — Length 131.1K | |||
| 20–21 | Kimi K2 (instruct, 2025-09) | Moonshot | Rating 59.1 | — | — | 262.1K | Moonshot Released 2025-09 Price — Length 262.1K | |||
| 22 | DeepSeek-R1 | DeepSeek | Rating 56.9 | — | — | 128K | DeepSeek Released 2025-01 Price — Length 128K | |||
| 23–24 | DeepSeek-V3 (2025-03) | DeepSeek | Rating 55.1 | — | — | 163.8K | DeepSeek Released 2025-03 Price — Length 163.8K | |||
| 23–24 | Gemini 2.5 Flash (preview, 2025-05) | Rating 55.1 | — | — | 1.05M | Google Released 2025-05 Price — Length 1.05M | ||||
| 25 | Grok 3 (beta) | xAI | Rating 53.3 | — | — | — | xAI Released 2025-04 Price — Length — | |||
| 26 | GPT-4.1 | OpenAI | Rating 52.4 | $2 | $8 | 1.05M | OpenAI Released 2025-04 Price $2 / $8 Length 1.05M | |||
| 27 | Claude 3.5 Sonnet (2024-10) | Anthropic | Rating 51.6 | — | — | 200K | Anthropic Released 2024-10 Price — Length 200K | |||
| 28 | Grok 3 Mini (beta) | xAI | Rating 49.3 | — | — | — | xAI Released 2025-04 Price — Length — | |||
| 29 | DeepSeek-V3 | DeepSeek | Rating 48.4 | — | — | 128K | DeepSeek Released 2024-12 Price — Length 128K | |||
| 30 | Gemini 2.5 Flash (preview, 2025-04) | Rating 47.1 | — | — | 1.05M | Google Released 2025-04 Price — Length 1.05M | ||||
| 31 | ChatGPT-4o (2025-03-27) | OpenAI | Rating 45.3 | — | — | — | OpenAI Released 2025-03 Price — Length — | |||
| 32 | GPT-4.5 (preview) | OpenAI | Rating 44.9 | — | — | — | OpenAI Released 2025-02 Price — Length — | |||
| 33 | GPT-OSS 120B | OpenAI | Rating 41.8 | — | — | 128K | OpenAI Released 2025-08 Price — Length 128K | |||
| 34 | Qwen3 32B | Alibaba | Rating 40.0 | $0.7 | $2.8 | 131.1K | Alibaba Released 2025-04 Price $0.7 / $2.8 Length 131.1K | |||
| 35 | Gemini Exp | Rating 38.2 | — | — | 2.1M | Google Released 2024-12 Price — Length 2.1M | ||||
| 36 | Gemini 2.0 Pro (exp) | Rating 35.6 | — | — | 2.1M | Google Released 2025-02 Price — Length 2.1M | ||||
| 37 | o1-mini | OpenAI | Rating 32.9 | — | — | 128K | OpenAI Released 2024-09 Price — Length 128K | |||
| 38 | GPT-4.1 mini | OpenAI | Rating 32.4 | — | — | 1.05M | OpenAI Released 2025-04 Price — Length 1.05M | |||
| 39 | Claude 3.5 Haiku | Anthropic | Rating 28.0 | — | — | 200K | Anthropic Released 2024-10 Price — Length 200K | |||
| 40 | ChatGPT-4o (2025-01) | OpenAI | Rating 27.1 | — | — | — | OpenAI Released 2025-01 Price — Length — | |||
| 41 | GPT-4o (2024-08) | OpenAI | Rating 23.1 | $2.5 | $10 | 128K | OpenAI Released 2024-08 Price $2.5 / $10 Length 128K | |||
| 42 | Gemini 2.0 Flash (thinking, exp) | Rating 22.2 | — | — | — | Google Released 2024-12 Price — Length — | ||||
| 43 | Qwen Max | Alibaba | Rating 21.8 | — | — | 128K | Alibaba Released 2025-01 Price — Length 128K | |||
| 44 | QwQ 32B | Alibaba | Rating 20.9 | — | — | 32.8K | Alibaba Released 2025-03 Price — Length 32.8K | |||
| 45 | GPT-4o (2024-11) | OpenAI | Rating 18.2 | $2.5 | $10 | 128K | OpenAI Released 2024-11 Price $2.5 / $10 Length 128K | |||
| 46 | DeepSeek-V2.5 | DeepSeek | Rating 17.8 | — | — | — | DeepSeek Released 2024-09 Price — Length — | |||
| 47 | Qwen2.5 Coder 32B (instruct) | Alibaba | Rating 16.4 | — | — | 32.8K | Alibaba Released 2024-11 Price — Length 32.8K | |||
| 48 | Llama 4 Maverick | Meta | Rating 15.6 | — | — | 524.3K | Meta Released 2025-04 Price — Length 524.3K | |||
| 49 | Yi Lightning | 01.AI | Rating 12.9 | — | — | — | 01.AI Released 2024-12 Price — Length — | |||
| 50 | Command A | Cohere | Rating 12.0 | $2.5 | $10 | 256K | Cohere Released 2025-03 Price $2.5 / $10 Length 256K | |||
| 51 | Codestral | Mistral | Rating 11.1 | — | — | 256K | Mistral Released 2025-01 Price — Length 256K | |||
| 52 | OpenHands LM 32B (v0.1) | All Hands AI | Rating 10.2 | — | — | — | — | All Hands AI Released 2024-03 Price — Length — | ||
| 53 | GPT-4.1 nano | OpenAI | Rating 8.9 | $0.1 | $0.4 | 1.05M | OpenAI Released 2025-04 Price $0.1 / $0.4 Length 1.05M | |||
| 54 | Gemma 3 27B | Rating 4.9 | — | — | 128K | Google Released 2025-03 Price — Length 128K | ||||
| 55 | GPT-4o mini | OpenAI | Rating 3.6 | — | — | 128K | OpenAI Released 2024-07 Price — Length 128K |
Best success rate over time
2024-03-26Release date2025-12-01
How to read this page
- It gets two tries at the tests, and the value is the share of problems that pass. Figures published by the organizations that made the tasks, among others, collected here.
- The success rate is the share of tries that succeeded. A gap of 1 point is 1 success out of 100 tries.
- In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
- Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
- Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.
Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-10-24, data sources