Task results
Research-level math problems (old version, FrontierMath Tier 4 2025-07)
The old version of FrontierMath Tier 4, a set of research-level problems harder than the base set.
Superseded by: FrontierMath Tier 4 v2
How to read this table
- Higher rows are rated higher. A range like “1–2” means the gap is within the margin of error.
- The score is the share of a shared set of problems the model solved (%).
- ± is how far the score moves when re-measured, so small gaps do not mean much.
- Prices are a guide per million tokens, reading then writing.
- Length is how much text the model can read at once (tokens).
± range of the top model
| Rank | Model | Developer | Accuracy (%) | Evaluated | Input $ /1M | Output $ /1M | Context | Supports | Weights | Released | Price and specs |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1–6 | AI co-mathematician | Rating 79.8 ±23.5 | — | — | — | Google Released 2026-05 Price — Length — | |||||
| 1–9 | GPT-5.5 Pro | OpenAI | Rating 66.0 ±23.2 | $30 | $180 | 1.05M | OpenAI Released 2026-04 Price $30 / $180 Length 1.05M | ||||
| 1–9 | GPT-5.4 Pro (web) | OpenAI | Rating 62.5 ±22.9 | — | — | — | OpenAI Released 2026-03 Price — Length — | ||||
| 1–11 | GPT-5.5 | OpenAI | Rating 59.0 ±22.5 | $5 | $30 | 1.05M | OpenAI Released 2026-04 Price $5 / $30 Length 1.05M | ||||
| 1–12 | GPT-5.2 Pro (web) | OpenAI | Rating 52.2 ±21.9 | — | — | — | OpenAI Released 2025-12 Price — Length — | ||||
| 1–12 | Claude Opus 4.8 | Anthropic | Rating 52.1 ±22.1 | $5 | $25 | 1M | Anthropic Released 2026-05 Price $5 / $25 Length 1M | ||||
| 2–19 | GPT-5.4 | OpenAI | Rating 45.2 ±20.9 | $2.5 | $15 | 1.05M | OpenAI Released 2026-03 Price $2.5 / $15 Length 1.05M | ||||
| 2–20 | Claude Opus 4.7 | Anthropic | Rating 38.2 ±20.0 | $5 | $25 | 1M | Anthropic Released 2026-04 Price $5 / $25 Length 1M | ||||
| 2–20 | Claude Opus 4.6 | Anthropic | Rating 38.2 ±19.9 | $5 | $25 | 1M | Anthropic Released 2026-02 Price $5 / $25 Length 1M | ||||
| 4–25 | GPT-5.2 | OpenAI | Rating 31.3 ±18.3 | $1.75 | $14 | 400K | OpenAI Released 2025-12 Price $1.75 / $14 Length 400K | ||||
| 4–25 | Gemini 3 Pro (preview) | Rating 31.3 ±18.6 | — | — | 1M | Google Released 2025-11 Price — Length 1M | |||||
| 5–25 | Gemini 3.1 Pro (preview) | Rating 27.8 ±17.6 | $2 | $12 | 1.05M | Google Released 2026-02 Price $2 / $12 Length 1.05M | |||||
| 7–35 | GPT-5 Pro | OpenAI | Rating 24.3 ±16.7 | $15 | $120 | 400K | OpenAI Released 2025-10 Price $15 / $120 Length 400K | ||||
| 7–35 | Muse Spark | Meta | Rating 24.3 ±16.7 | — | — | — | Meta Released 2026-04 Price — Length — | ||||
| 7–35 | Gemini 3.5 Flash | Rating 24.3 ±16.8 | $1.5 | $9 | 1.05M | Google Released 2026-05 Price $1.5 / $9 Length 1.05M | |||||
| 7–35 | Kimi K2.6 | Moonshot | Rating 24.3 ±16.8 | $0.95 | $4 | 262.1K | Moonshot Released 2026-04 Price $0.95 / $4 Length 262.1K | ||||
| 7–35 | GLM-5.1 | Z.ai | Rating 20.8 ±15.8 | $1.4 | $4.4 | 200K | Z.ai Released 2026-04 Price $1.4 / $4.4 Length 200K | ||||
| 7–35 | GPT-5.1 | OpenAI | Rating 20.8 ±15.8 | $1.25 | $10 | 400K | OpenAI Released 2025-11 Price $1.25 / $10 Length 400K | ||||
| 7–35 | GPT-5 | OpenAI | Rating 20.8 ±15.8 | $1.25 | $10 | 400K | OpenAI Released 2025-08 Price $1.25 / $10 Length 400K | ||||
| 8–46 | Gemini 2.5 Deep Think (web) | Rating 17.3 ±14.4 | — | — | — | — | Google Released 2025-08 Price — Length — | ||||
| 10–46 | Qwen3.6 Plus | Alibaba | Rating 13.9 ±13.2 | $0.5 | $3 | 1M | Alibaba Released 2026-03 Price $0.5 / $3 Length 1M | ||||
| 10–46 | Claude Sonnet 4.6 | Anthropic | Rating 13.8 ±13.1 | $3 | $15 | 1M | Anthropic Released 2026-02 Price $3 / $15 Length 1M | ||||
| 10–56 | GPT-5.4 nano | OpenAI | Rating 10.4 ±11.5 | $0.2 | $1.25 | 400K | OpenAI Released 2026-03 Price $0.2 / $1.25 Length 400K | ||||
| 10–56 | GPT-5 mini | OpenAI | Rating 10.4 ±11.5 | $0.25 | $2 | 400K | OpenAI Released 2025-08 Price $0.25 / $2 Length 400K | ||||
| 10–56 | o4-mini | OpenAI | Rating 10.4 ±11.5 | $1.1 | $4.4 | 200K | OpenAI Released 2025-04 Price $1.1 / $4.4 Length 200K | ||||
| 13–56 | Kimi K2.5 | Moonshot | Rating 7.0 ±9.5 | — | — | 262.1K | Moonshot Released 2026-01 Price — Length 262.1K | ||||
| 13–56 | Claude Opus 4.1 | Anthropic | Rating 6.9 ±9.5 | — | — | 200K | Anthropic Released 2025-08 Price — Length 200K | ||||
| 13–56 | Claude Opus 4 | Anthropic | Rating 6.9 ±9.5 | — | — | 200K | Anthropic Released 2025-05 Price — Length 200K | ||||
| 13–56 | Claude Opus 4.5 | Anthropic | Rating 6.9 ±9.5 | $5 | $25 | 200K | Anthropic Released 2025-11 Price $5 / $25 Length 200K | ||||
| 13–56 | Claude Sonnet 4.5 | Anthropic | Rating 6.9 ±9.5 | $3 | $15 | 200K | Anthropic Released 2025-09 Price $3 / $15 Length 200K | ||||
| 13–56 | Gemini 2.5 Flash | Rating 6.9 ±9.5 | $0.3 | $2.5 | 1M | Google Released 2025-06 Price $0.3 / $2.5 Length 1M | |||||
| 13–56 | Gemini 2.5 Pro | Rating 6.9 ±9.5 | $1.25 | $10 | 1M | Google Released 2025-06 Price $1.25 / $10 Length 1M | |||||
| 13–56 | Gemini 3 Flash (preview) | Rating 6.9 ±9.5 | $0.5 | $3 | 1M | Google Released 2025-12 Price $0.5 / $3 Length 1M | |||||
| 13–56 | o3-mini | OpenAI | Rating 6.9 ±9.5 | $1.1 | $4.4 | 200K | OpenAI Released 2025-01 Price $1.1 / $4.4 Length 200K | ||||
| 13–56 | Qwen3.6 Max (preview) | Alibaba | Rating 6.9 ±9.5 | $1.3 | $7.8 | 240K | Alibaba Released 2026-04 Price $1.3 / $7.8 Length 240K | ||||
| 20–56 | GLM-4.6 | Z.ai | Rating 3.5 ±7.0 | $0.6 | $2.2 | 204.8K | Z.ai Released 2025-09 Price $0.6 / $2.2 Length 204.8K | ||||
| 20–56 | DeepSeek-V3.2 | DeepSeek | Rating 3.5 ±6.9 | — | — | 128K | DeepSeek Released 2025-12 Price — Length 128K | ||||
| 20–56 | GLM-5 | Z.ai | Rating 3.5 ±6.9 | $1 | $3.2 | 204.8K | Z.ai Released 2026-02 Price $1 / $3.2 Length 204.8K | ||||
| 20–56 | Claude Haiku 4.5 | Anthropic | Rating 3.5 ±6.8 | $1 | $5 | 200K | Anthropic Released 2025-10 Price $1 / $5 Length 200K | ||||
| 20–56 | Gemini 2.5 Pro (preview, 2025-05) | Rating 3.5 ±6.8 | — | — | 1.05M | Google Released 2025-05 Price — Length 1.05M | |||||
| 20–56 | GPT-5 nano | OpenAI | Rating 3.5 ±6.8 | $0.05 | $0.4 | 400K | OpenAI Released 2025-08 Price $0.05 / $0.4 Length 400K | ||||
| 20–56 | Grok 4 | xAI | Rating 3.5 ±6.8 | — | — | 256K | xAI Released 2025-07 Price — Length 256K | ||||
| 20–56 | o3 | OpenAI | Rating 3.5 ±6.8 | $2 | $8 | 200K | OpenAI Released 2025-04 Price $2 / $8 Length 200K | ||||
| 20–56 | Qwen3.5 Plus | Alibaba | Rating 3.5 ±6.8 | $0.4 | $2.4 | 1M | Alibaba Released 2026-02 Price $0.4 / $2.4 Length 1M | ||||
| 20–56 | GPT-5.4 mini | OpenAI | Rating 3.5 ±6.8 | $0.75 | $4.5 | 400K | OpenAI Released 2026-03 Price $0.75 / $4.5 Length 400K | ||||
| 20–56 | Grok 4 Heavy (web) | xAI | Rating 3.5 ±6.9 | — | — | — | xAI Released 2025-07 Price — Length — | ||||
| 23–56 | Claude 3.5 Sonnet (2024-06) | Anthropic | Rating 0.0 ±0.0 | — | — | 200K | Anthropic Released 2024-06 Price — Length 200K | ||||
| 23–56 | Claude 3.5 Sonnet (2024-10) | Anthropic | Rating 0.0 ±0.0 | — | — | 200K | Anthropic Released 2024-10 Price — Length 200K | ||||
| 23–56 | Claude Sonnet 4 | Anthropic | Rating 0.0 ±0.0 | — | — | 200K | Anthropic Released 2025-05 Price — Length 200K | ||||
| 23–56 | GPT-4.1 | OpenAI | Rating 0.0 ±0.0 | $2 | $8 | 1.05M | OpenAI Released 2025-04 Price $2 / $8 Length 1.05M | ||||
| 23–56 | Grok 3 (beta) | xAI | Rating 0.0 ±0.0 | — | — | — | xAI Released 2025-04 Price — Length — | ||||
| 23–56 | Kimi K2 (thinking) | Moonshot | Rating 0.0 ±0.0 | — | — | 262.1K | Moonshot Released 2025-11 Price — Length 262.1K | ||||
| 23–56 | Qwen3 235B A22B (thinking) | Alibaba | Rating 0.0 ±0.0 | — | — | 262.1K | Alibaba Released 2025-07 Price — Length 262.1K | ||||
| 23–56 | Qwen3.5 Flash | Alibaba | Rating 0.0 ±0.0 | $0.1 | $0.4 | 1M | Alibaba Released 2026-02 Price $0.1 / $0.4 Length 1M | ||||
| 23–56 | Qwen3.6 Flash | Alibaba | Rating 0.0 ±0.0 | $0.25 | $1.5 | 1M | Alibaba Released 2026-04 Price $0.25 / $1.5 Length 1M | ||||
| 23–56 | GLM-4.7 | Z.ai | Rating 0.0 ±0.0 | $0.6 | $2.2 | 204.8K | Z.ai Released 2025-12 Price $0.6 / $2.2 Length 204.8K |
Best accuracy over time
2024-06-20Release date2026-05-28
How to read this page
- Its successor is Tier 4 v2, which fixed problems with errors. The value is accuracy.
- In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
- Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
- Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.
Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-10-24, data sources