Reported by others
Simulated company work (The Agent Company)
In an environment modeled on a software company, the model uses internal tools and messages simulated coworkers to finish engineering, administrative and finance tasks.
How to read this table
- Models near the top are rated higher.
- The score is the value published by the task's authors and other organizations.
- Conditions can differ between models, so small gaps do not mean much.
- Prices are a guide per million tokens, reading then writing.
- Length is how much text the model can read at once (tokens).
| Rank | Model | Developer | Success rate (%) | Input $ /1M | Output $ /1M | Context | Supports | Weights | Released | Price and specs |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V3.2 (exp) | DeepSeek | Rating 42.9 | — | — | 163.8K | DeepSeek Released 2025-09 Price — Length 163.8K | |||
| 2 | Gemini 2.5 Flash (preview, 2025-09) | Rating 41.1 | — | — | 1M | Google Released 2025-09 Price — Length 1M | ||||
| 3 | Claude Sonnet 4 | Anthropic | Rating 33.1 | — | — | 200K | Anthropic Released 2025-05 Price — Length 200K | |||
| 4 | Claude 3.7 Sonnet | Anthropic | Rating 30.9 | — | — | 200K | Anthropic Released 2025-02 Price — Length 200K | |||
| 5 | Gemini 2.5 Pro (preview, 2025-05) | Rating 30.3 | — | — | 1.05M | Google Released 2025-05 Price — Length 1.05M | ||||
| 6 | Claude 3.5 Sonnet (2024-10) | Anthropic | Rating 24.0 | — | — | 200K | Anthropic Released 2024-10 Price — Length 200K | |||
| 7 | Gemini 2.0 Flash | Rating 11.4 | — | — | — | Google Released 2025-02 Price — Length — | ||||
| 8 | GPT-4o (2024-11) | OpenAI | Rating 8.6 | $2.5 | $10 | 128K | OpenAI Released 2024-11 Price $2.5 / $10 Length 128K | |||
| 9 | Llama 3.1 405B (instruct) | Meta | Rating 7.4 | — | — | — | Meta Released 2024-07 Price — Length — | |||
| 10 | Llama 3.1 70B (instruct) | Meta | Rating 6.9 | — | — | 128K | Meta Released 2024-07 Price — Length 128K | |||
| 11 | Qwen2.5 72B (instruct) | Alibaba | Rating 5.7 | — | — | 128K | Alibaba Released 2024-09 Price — Length 128K | |||
| 12 | Gemini 1.5 Pro 002 | Rating 3.4 | — | — | — | Google Released 2024-09 Price — Length — | ||||
| 13 | Amazon Nova Pro 1.0 | Amazon | Rating 1.7 | — | — | 300K | Amazon Released 2024-12 Price — Length 300K | |||
| 14 | Qwen2 72B (instruct) | Alibaba | Rating 1.1 | — | — | — | Alibaba Released 2024-06 Price — Length — |
Best success rate over time
2024-06-07Release date2025-09-29
How to read this page
- The value is the share of tasks finished. Figures published by the organizations that made the tasks, among others, collected here.
- The success rate is the share of tries that succeeded. A gap of 1 point is 1 success out of 100 tries.
- In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
- Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
- Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.
Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-10-24, data sources