Task results
Reimplementing whole programs, long tasks (MirrorCode)
Tasks where the model reimplements an entire existing program from scratch without seeing the original source code.
How to read this table
- Higher rows are rated higher. A range like “1–2” means the gap is within the margin of error.
- The score is the share of tries on a shared set of tasks that the model completed (%).
- A gap of 3 means 3 successes in 100 tries; ± is how far the score moves when re-measured.
- Prices are a guide per million tokens, reading then writing.
- Length is how much text the model can read at once (tokens).
± range of the top model
| Rank | Model | Developer | Success rate (%) | Evaluated | Input $ /1M | Output $ /1M | Context | Supports | Weights | Released | Price and specs |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1–4 | Claude Opus 5.5 | Anthropic | Rating 77.4 ±16.3 | $4 | $20 | 1M | Anthropic Released 2026-09 Price $4 / $20 Length 1M | ||||
| 1–5 | Claude Fable 5.1 | Anthropic | Rating 73.3 ±18.0 | $10 | $50 | 1M | Anthropic Released 2026-09 Price $10 / $50 Length 1M | ||||
| 1–5 | Claude Fable 5 | Anthropic | Rating 63.9 ±20.4 | $10 | $50 | 1M | Anthropic Released 2026-06 Price $10 / $50 Length 1M | ||||
| 1–6 | Gemini 4 Argon | Rating 58.9 ±23.0 | $2 | $10 | 1.05M | Google Released 2026-09 Price $2 / $10 Length 1.05M | |||||
| 2–7 | GPT-6 Astra | OpenAI | Rating 46.7 ±23.2 | $10 | $50 | 1.05M | OpenAI Released 2026-09 Price $10 / $50 Length 1.05M | ||||
| 4–9 | Claude Opus 4.7 | Anthropic | Rating 31.1 ±17.7 | $5 | $25 | 1M | Anthropic Released 2026-04 Price $5 / $25 Length 1M | ||||
| 5–10 | GPT-5.6 Sol | OpenAI | Rating 20.0 ±18.1 | $4 | $20 | 1.05M | OpenAI Released 2026-07 Price $4 / $20 Length 1.05M | ||||
| 6–10 | GPT-5.4 | OpenAI | Rating 15.6 ±15.1 | $2.5 | $15 | 1.05M | OpenAI Released 2026-03 Price $2.5 / $15 Length 1.05M | ||||
| 6–10 | GPT-5.5 | OpenAI | Rating 10.0 ±11.8 | $5 | $30 | 1.05M | OpenAI Released 2026-04 Price $5 / $30 Length 1.05M | ||||
| 7–10 | Gemini 3.1 Pro (preview) | Rating 8.9 ±8.9 | $2 | $12 | 1.05M | Google Released 2026-02 Price $2 / $12 Length 1.05M |
Best success rate over time
2026-02-19Release date2026-09-30
How to read this page
- The reimplementation is graded on whether its output exactly matches the original program, including on held-out tests, and the value is the share of tasks that pass all tests.
- The success rate is the share of tries that succeeded. A gap of 1 point is 1 success out of 100 tries.
- In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
- Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
- Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.
Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-10-24, data sources