Task results
Math competition problems, top difficulty level (MATH level 5)
Only the top difficulty level of the MATH dataset, which collects problems from math competitions such as the AMC 10, AMC 12, and AIME.
How to read this table
- Higher rows are rated higher. A range like “1–2” means the gap is within the margin of error.
- The score is the share of a shared set of problems the model solved (%).
- A gap of 3 means 3 problems out of 100, and ± is how far the score moves when re-measured.
- Prices are a guide per million tokens, reading then writing.
- Length is how much text the model can read at once (tokens).
± range of the top model
| Rank | Model | Developer | Accuracy (%) | Evaluated | Input $ /1M | Output $ /1M | Context | Supports | Weights | Released | Price and specs |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1–6 | GPT-5 | OpenAI | Rating 98.1 ±0.6 | $1.25 | $10 | 400K | OpenAI Released 2025-08 Price $1.25 / $10 Length 400K | ||||
| 1–6 | GPT-5 mini | OpenAI | Rating 97.8 ±0.7 | $0.25 | $2 | 400K | OpenAI Released 2025-08 Price $0.25 / $2 Length 400K | ||||
| 1–6 | o4-mini | OpenAI | Rating 97.8 ±0.7 | $1.1 | $4.4 | 200K | OpenAI Released 2025-04 Price $1.1 / $4.4 Length 200K | ||||
| 1–6 | o3 | OpenAI | Rating 97.8 ±0.7 | $2 | $8 | 200K | OpenAI Released 2025-04 Price $2 / $8 Length 200K | ||||
| 1–7 | Claude Sonnet 4.5 | Anthropic | Rating 97.7 ±0.8 | $3 | $15 | 200K | Anthropic Released 2025-09 Price $3 / $15 Length 200K | ||||
| 1–9 | Qwen3 Max (instruct) | Alibaba | Rating 97.1 ±0.8 | — | — | 258K | Alibaba Released 2025-09 Price — Length 258K | ||||
| 5–11 | DeepSeek-R1 (2025-05) | DeepSeek | Rating 96.6 ±0.8 | — | — | 131.1K | DeepSeek Released 2025-05 Price — Length 131.1K | ||||
| 6–11 | o3-mini | OpenAI | Rating 96.5 ±0.8 | $1.1 | $4.4 | 200K | OpenAI Released 2025-01 Price $1.1 / $4.4 Length 200K | ||||
| 6–12 | Claude Haiku 4.5 | Anthropic | Rating 96.4 ±1.0 | $1 | $5 | 200K | Anthropic Released 2025-10 Price $1 / $5 Length 200K | ||||
| 7–13 | Gemini 2.5 Pro (preview, 2025-05) | Rating 95.9 ±0.8 | — | — | 1.05M | Google Released 2025-05 Price — Length 1.05M | |||||
| 7–13 | Gemini 2.5 Pro (preview, 2025-03) | Rating 95.6 ±0.9 | — | — | 1.05M | Google Released 2025-03 Price — Length 1.05M | |||||
| 9–13 | GPT-5 nano | OpenAI | Rating 95.2 ±0.9 | $0.05 | $0.4 | 400K | OpenAI Released 2025-08 Price $0.05 / $0.4 Length 400K | ||||
| 10–14 | o1 | OpenAI | Rating 94.7 ±1.2 | $15 | $60 | 128K | OpenAI Released 2024-12 Price $15 / $60 Length 128K | ||||
| 13–15 | DeepSeek-R1 | DeepSeek | Rating 93.1 ±1.4 | — | — | 128K | DeepSeek Released 2025-01 Price — Length 128K | ||||
| 14–18 | Claude 3.7 Sonnet | Anthropic | Rating 91.2 ±1.5 | — | — | 200K | Anthropic Released 2025-02 Price — Length 200K | ||||
| 15–19 | Grok 3 Mini (beta) | xAI | Rating 90.9 ±1.5 | — | — | — | xAI Released 2025-04 Price — Length — | ||||
| 15–19 | DeepSeek-R1-Distill-Llama-70B | DeepSeek | Rating 89.9 ±1.3 | — | — | 32.8K | DeepSeek Released 2025-01 Price — Length 32.8K | ||||
| 15–20 | o1-mini | OpenAI | Rating 89.2 ±1.3 | — | — | 128K | OpenAI Released 2024-09 Price — Length 128K | ||||
| 16–21 | Grok 3 (beta) | xAI | Rating 88.7 ±1.7 | — | — | — | xAI Released 2025-04 Price — Length — | ||||
| 18–22 | GPT-4.1 mini | OpenAI | Rating 87.3 ±1.4 | — | — | 1.05M | OpenAI Released 2025-04 Price — Length 1.05M | ||||
| 19–22 | DeepSeek-R1-Distill-Qwen-14B | DeepSeek | Rating 87.1 ±1.5 | — | — | 32.8K | DeepSeek Released 2025-01 Price — Length 32.8K | ||||
| 20–25 | Claude Opus 4 | Anthropic | Rating 85.0 ±1.9 | — | — | 200K | Anthropic Released 2025-05 Price — Length 200K | ||||
| 22–27 | Claude Sonnet 4 | Anthropic | Rating 84.4 ±2.0 | — | — | 200K | Anthropic Released 2025-05 Price — Length 200K | ||||
| 22–28 | Gemini 2.0 Pro (exp) | Rating 83.5 ±2.0 | — | — | 2.1M | Google Released 2025-02 Price — Length 2.1M | |||||
| 22–28 | GPT-4.1 | OpenAI | Rating 83.0 ±1.7 | $2 | $8 | 1.05M | OpenAI Released 2025-04 Price $2 / $8 Length 1.05M | ||||
| 23–28 | Gemini 2.0 Flash | Rating 82.2 ±1.7 | — | — | — | Google Released 2025-02 Price — Length — | |||||
| 23–29 | o1-preview | OpenAI | Rating 81.6 ±2.1 | — | — | — | OpenAI Released 2024-09 Price — Length — | ||||
| 24–28 | Mistral Medium 3 | Mistral | Rating 81.6 ±1.8 | $0.4 | $2 | 128K | Mistral Released 2025-05 Price $0.4 / $2 Length 128K | ||||
| 28–29 | GPT-4.5 (preview) | OpenAI | Rating 78.6 ±2.2 | — | — | — | OpenAI Released 2025-02 Price — Length — | ||||
| 30–32 | DeepSeek-V3 (2025-03) | DeepSeek | Rating 75.5 ±1.7 | — | — | 163.8K | DeepSeek Released 2025-03 Price — Length 163.8K | ||||
| 30–32 | Gemma 3 27B | Rating 74.0 ±2.2 | — | — | 128K | Google Released 2025-03 Price — Length 128K | |||||
| 30–33 | Llama 4 Maverick (FP8) | Meta | Rating 73.0 ±2.1 | — | — | 1.05M | Meta Released 2025-04 Price — Length 1.05M | ||||
| 32–35 | Gemini 1.5 Pro 002 | Rating 70.4 ±2.0 | — | — | — | Google Released 2024-09 Price — Length — | |||||
| 33–36 | GPT-4.1 nano | OpenAI | Rating 70.0 ±2.0 | $0.1 | $0.4 | 1.05M | OpenAI Released 2025-04 Price $0.1 / $0.4 Length 1.05M | ||||
| 33–36 | Qwen3 235B A22B | Alibaba | Rating 68.9 ±1.9 | $0.7 | $2.8 | 128K | Alibaba Released 2025-04 Price $0.7 / $2.8 Length 128K | ||||
| 34–39 | Qwen Max | Alibaba | Rating 67.2 ±2.2 | — | — | 128K | Alibaba Released 2025-01 Price — Length 128K | ||||
| 36–42 | Qwen Plus (2025-01) | Alibaba | Rating 65.3 ±2.2 | — | — | — | Alibaba Released 2025-01 Price — Length — | ||||
| 36–42 | Phi-4 | Microsoft | Rating 64.9 ±2.1 | — | — | 128K | Microsoft Released 2024-12 Price — Length 128K | ||||
| 36–43 | DeepSeek-V3 | DeepSeek | Rating 64.9 ±2.1 | — | — | 128K | DeepSeek Released 2024-12 Price — Length 128K | ||||
| 37–43 | Grok 2 (2024-12) | xAI | Rating 63.5 ±2.6 | — | — | — | xAI Released 2024-12 Price — Length — | ||||
| 37–43 | Qwen2.5 72B (instruct) | Alibaba | Rating 63.2 ±2.1 | — | — | 128K | Alibaba Released 2024-09 Price — Length 128K | ||||
| 37–43 | Llama 4 Scout | Meta | Rating 62.3 ±2.4 | — | — | 128K | Meta Released 2025-04 Price — Length 128K | ||||
| 39–43 | Gemini 1.5 Flash 002 | Rating 61.9 ±2.2 | — | — | — | Google Released 2024-09 Price — Length — | |||||
| 44–46 | Claude 3.5 Sonnet (2024-10) | Anthropic | Rating 56.9 ±2.2 | — | — | 200K | Anthropic Released 2024-10 Price — Length 200K | ||||
| 44–47 | Qwen Turbo | Alibaba | Rating 56.2 ±2.2 | — | — | — | Alibaba Released 2024-11 Price — Length — | ||||
| 44–47 | Qwen2.5 32B (instruct) | Alibaba | Rating 56.1 ±2.1 | — | — | — | Alibaba Released 2024-09 Price — Length — | ||||
| 45–51 | GPT-4o (2024-08) | OpenAI | Rating 53.3 ±2.2 | $2.5 | $10 | 128K | OpenAI Released 2024-08 Price $2.5 / $10 Length 128K | ||||
| 47–53 | GPT-4o mini | OpenAI | Rating 52.6 ±2.2 | — | — | 128K | OpenAI Released 2024-07 Price — Length 128K | ||||
| 47–53 | Claude 3.5 Sonnet (2024-06) | Anthropic | Rating 51.7 ±2.3 | — | — | 200K | Anthropic Released 2024-06 Price — Length 200K | ||||
| 47–53 | GPT-4o (2024-05) | OpenAI | Rating 51.0 ±2.2 | $5 | $15 | 128K | OpenAI Released 2024-05 Price $5 / $15 Length 128K | ||||
| 47–53 | Mistral Large (2024-11) | Mistral | Rating 50.3 ±2.2 | — | — | 128K | Mistral Released 2024-11 Price — Length 128K | ||||
| 48–55 | GPT-4o (2024-11) | OpenAI | Rating 49.8 ±2.2 | $2.5 | $10 | 128K | OpenAI Released 2024-11 Price $2.5 / $10 Length 128K | ||||
| 48–55 | Llama 3.1 405B (instruct) | Meta | Rating 49.8 ±2.4 | — | — | — | Meta Released 2024-07 Price — Length — | ||||
| 52–58 | Mistral Small 3.1 | Mistral | Rating 46.8 ±2.4 | — | — | 131.1K | Mistral Released 2025-03 Price — Length 131.1K | ||||
| 52–58 | GPT-4 Turbo | OpenAI | Rating 46.7 ±2.2 | — | — | — | OpenAI Released 2024-04 Price — Length — | ||||
| 54–58 | Claude 3.5 Haiku | Anthropic | Rating 46.4 ±2.3 | — | — | 200K | Anthropic Released 2024-10 Price — Length 200K | ||||
| 54–59 | Mistral Large 2 | Mistral | Rating 44.8 ±2.1 | — | — | 131.1K | Mistral Released 2024-07 Price — Length 131.1K | ||||
| 54–59 | Mistral Small 3 | Mistral | Rating 44.8 ±2.1 | — | — | 32.8K | Mistral Released 2025-01 Price — Length 32.8K | ||||
| 57–62 | Llama 3.1 Tulu 3 70B DPO | Ai2 | Rating 42.7 ±2.0 | — | — | — | Ai2 Released 2024-11 Price — Length — | ||||
| 59–64 | Llama 3.3 70B (instruct) | Meta | Rating 41.6 ±2.3 | — | — | 131.1K | Meta Released 2024-12 Price — Length 131.1K | ||||
| 59–64 | Gemini 1.5 Pro 001 | Rating 40.7 ±2.1 | — | — | — | Google Released 2024-05 Price — Length — | |||||
| 59–65 | GPT-4 Turbo (preview, 2023-11) | OpenAI | Rating 40.0 ±2.1 | — | — | — | OpenAI Released 2023-11 Price — Length — | ||||
| 60–66 | Llama 3.2 90B Vision (instruct) | Meta | Rating 39.4 ±2.0 | — | — | 16K | Meta Released 2024-09 Price — Length 16K | ||||
| 60–66 | Qwen2 72B (instruct) | Alibaba | Rating 39.1 ±2.0 | — | — | — | Alibaba Released 2024-06 Price — Length — | ||||
| 62–67 | Claude 3 Opus | Anthropic | Rating 37.5 ±2.1 | — | — | — | Anthropic Released 2024-02 Price — Length — | ||||
| 63–67 | Llama 3.1 70B (instruct) | Meta | Rating 36.7 ±2.0 | — | — | 128K | Meta Released 2024-07 Price — Length 128K | ||||
| 65–67 | GPT-4 Turbo (preview, 2024-01) | OpenAI | Rating 35.4 ±2.1 | — | — | — | OpenAI Released 2024-01 Price — Length — | ||||
| 68–70 | Gemma 2 27B | Rating 27.9 ±1.9 | — | — | 8.2K | Google Released 2024-06 Price — Length 8.2K | |||||
| 68–73 | WizardLM-2-8x22B | Microsoft | Rating 25.7 ±1.7 | — | — | 65.5K | Microsoft Released 2024-04 Price — Length 65.5K | ||||
| 68–73 | Yi 1.5 34B (chat) | 01.AI | Rating 25.5 ±1.7 | — | — | — | 01.AI Released 2024-05 Price — Length — | ||||
| 69–76 | Gemini 1.5 Flash | Rating 25.1 ±1.9 | — | — | — | Google Released 2024-05 Price — Length — | |||||
| 69–77 | Mistral Large (2024-02) | Mistral | Rating 24.5 ±1.7 | — | — | 32K | Mistral Released 2024-02 Price — Length 32K | ||||
| 69–77 | Open Mixtral 8x22B | Mistral | Rating 24.2 ±1.6 | — | — | — | Mistral Released 2024-04 Price — Length — | ||||
| 71–78 | GPT-4 (2023-06) | OpenAI | Rating 23.0 ±1.7 | — | — | — | OpenAI Released 2023-06 Price — Length — | ||||
| 71–78 | Llama 3.1 8B (instruct) | Meta | Rating 22.9 ±1.7 | — | — | 128K | Meta Released 2024-07 Price — Length 128K | ||||
| 71–78 | Hermes-2-Theta-Llama-3-70B | Nous Research | Rating 22.7 ±1.6 | — | — | — | Nous Research Released 2024-06 Price — Length — | ||||
| 72–78 | Llama 3 70B (instruct) | Meta | Rating 22.6 ±1.6 | — | — | 8.2K | Meta Released 2024-04 Price — Length 8.2K | ||||
| 74–78 | Gemma 2 9B | Rating 21.0 ±1.8 | — | — | — | Google Released 2024-06 Price — Length — | |||||
| 79–80 | Claude 3 Sonnet | Anthropic | Rating 18.2 ±1.6 | — | — | — | Anthropic Released 2024-02 Price — Length — | ||||
| 79–81 | Phi-3 medium (128k, instruct) | Microsoft | Rating 17.6 ±1.4 | — | — | — | Microsoft Released 2024-04 Price — Length — | ||||
| 80–84 | GPT-3.5 Turbo (2023-11) | OpenAI | Rating 15.9 ±1.4 | — | — | 16.4K | OpenAI Released 2023-11 Price — Length 16.4K | ||||
| 81–84 | Ministral 8B | Mistral | Rating 14.9 ±1.5 | — | — | — | Mistral Released 2024-10 Price — Length — | ||||
| 81–84 | Claude 3 Haiku | Anthropic | Rating 14.9 ±1.4 | — | — | 200K | Anthropic Released 2024-03 Price — Length 200K | ||||
| 81–84 | Ministral 3B | Mistral | Rating 14.4 ±1.5 | — | — | — | Mistral Released 2024-10 Price — Length — | ||||
| 85–89 | Claude 2.0 | Anthropic | Rating 11.7 ±1.3 | — | — | — | Anthropic Released 2023-07 Price — Length — | ||||
| 85–90 | DBRX (instruct) | Databricks | Rating 11.7 ±1.4 | — | — | — | Databricks Released 2024-03 Price — Length — | ||||
| 85–89 | GPT-3.5 Turbo (2024-01) | OpenAI | Rating 11.6 ±1.1 | — | — | 16.4K | OpenAI Released 2024-01 Price — Length 16.4K | ||||
| 85–90 | Gemini 1.0 Pro 001 | Rating 11.2 ±1.2 | — | — | — | Google Released 2023-12 Price — Length — | |||||
| 85–91 | Open Mistral Nemo | Mistral | Rating 10.8 ±1.2 | — | — | — | Mistral Released 2024-07 Price — Length — | ||||
| 87–91 | Open Mixtral 8x7B | Mistral | Rating 10.0 ±1.0 | — | — | — | Mistral Released 2023-12 Price — Length — | ||||
| 89–91 | Mixtral 8x7B (instruct, v0.1) | Mistral | Rating 9.3 ±1.0 | — | — | 32K | Mistral Released 2023-12 Price — Length 32K | ||||
| 92–93 | DeepSeek LLM 67B (chat) | DeepSeek | Rating 6.4 ±0.8 | — | — | — | DeepSeek Released 2023-11 Price — Length — | ||||
| 92–94 | Llama 3 8B (instruct) | Meta | Rating 6.1 ±0.7 | — | — | — | Meta Released 2024-04 Price — Length — | ||||
| 93–94 | Yi 34B (chat) | 01.AI | Rating 5.1 ±0.7 | — | — | — | 01.AI Released 2023-11 Price — Length — | ||||
| 95–97 | Open Mistral 7B | Mistral | Rating 3.7 ±0.5 | — | — | — | Mistral Released 2023-09 Price — Length — | ||||
| 95–97 | Mistral 7B (instruct, v0.3) | Mistral | Rating 3.6 ±0.5 | — | — | 65.5K | Mistral Released 2024-05 Price — Length 65.5K | ||||
| 95–97 | Llama 2 70B (chat) | Meta | Rating 3.3 ±0.5 | — | — | — | Meta Released 2023-07 Price — Length — |
Best accuracy over time
2023-06-13Release date2025-10-15
How to read this page
- Answers must follow a fixed format, and the value is accuracy.
- The accuracy is the share of problems solved. A gap of 1 point is 1 question out of 100.
- In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
- Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
- Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.
Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-10-24, data sources