Reported by others
The hardest academic questions (HLE)
Questions written by experts in math, the sciences, the humanities and many other fields, kept only if models at the time could not solve them.
Updates to this data stopped on 2026-10-24 (last values shown)
How to read this table
- Higher rows are rated higher. A range like “1–2” means the gap is within the margin of error.
- The score is the value published by the task's authors and other organizations.
- Conditions can differ between models, so small gaps do not mean much.
- Prices are a guide per million tokens, reading then writing.
- Length is how much text the model can read at once (tokens).
± range of the top model
| Rank | Model | Developer | Accuracy (%) | Input $ /1M | Output $ /1M | Context | Supports | Weights | Released | Price and specs |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | OpenAI | Rating 52.5 ±4.0 | $10 | $50 | 1.05M | OpenAI Released 2026-09 Price $10 / $50 Length 1.05M | |||
| 2–6 | Claude Fable 5.1 | Anthropic | Rating 43.8 ±4.1 | $10 | $50 | 1M | Anthropic Released 2026-09 Price $10 / $50 Length 1M | |||
| 2–6 | Gemini 3.1 Pro (preview) | Rating 43.7 ±4.0 | $2 | $12 | 1.05M | Google Released 2026-02 Price $2 / $12 Length 1.05M | ||||
| 2–6 | Claude Sonnet 5.5 | Anthropic | Rating 43.4 ±4.0 | $2 | $10 | 1M | Anthropic Released 2026-09 Price $2 / $10 Length 1M | |||
| 2–7 | Gemini 3.8 Flash | Rating 41.7 ±4.0 | $0.75 | $3.75 | 1.05M | Google Released 2026-09 Price $0.75 / $3.75 Length 1.05M | ||||
| 2–7 | GPT-5.4 Pro | OpenAI | Rating 41.5 ±4.0 | $30 | $180 | 1.05M | OpenAI Released 2026-03 Price $30 / $180 Length 1.05M | |||
| 5–10 | Muse Spark | Meta | Rating 37.6 ±4.0 | — | — | — | Meta Released 2026-04 Price — Length — | |||
| 7–11 | Gemini 3 Pro (preview) | Rating 34.4 ±3.9 | — | — | 1M | Google Released 2025-11 Price — Length 1M | ||||
| 7–12 | GPT-5.4 | OpenAI | Rating 33.0 ±3.9 | $2.5 | $15 | 1.05M | OpenAI Released 2026-03 Price $2.5 / $15 Length 1.05M | |||
| 7–12 | Claude Opus 4.7 | Anthropic | Rating 33.0 ±3.9 | $5 | $25 | 1M | Anthropic Released 2026-04 Price $5 / $25 Length 1M | |||
| 8–13 | Claude Opus 4.6 | Anthropic | Rating 31.1 ±3.8 | $5 | $25 | 1M | Anthropic Released 2026-02 Price $5 / $25 Length 1M | |||
| 9–14 | GPT-5 Pro | OpenAI | Rating 28.2 ±3.7 | $15 | $120 | 400K | OpenAI Released 2025-10 Price $15 / $120 Length 400K | |||
| 11–15 | Claude Haiku 5.5 | Anthropic | Rating 26.6 ±3.8 | $0.1 | $0.5 | 1M | Anthropic Released 2026-10 Price $0.1 / $0.5 Length 1M | |||
| 12–18 | GPT-5.2 | OpenAI | Rating 24.2 ±3.6 | $1.75 | $14 | 400K | OpenAI Released 2025-12 Price $1.75 / $14 Length 400K | |||
| 13–19 | GPT-5 | OpenAI | Rating 21.6 ±3.5 | $1.25 | $10 | 400K | OpenAI Released 2025-08 Price $1.25 / $10 Length 400K | |||
| 14–19 | Claude Opus 4.5 | Anthropic | Rating 21.4 ±3.5 | $5 | $25 | 200K | Anthropic Released 2025-11 Price $5 / $25 Length 200K | |||
| 14–20 | Kimi K2.5 | Moonshot | Rating 20.6 ±3.7 | — | — | 262.1K | Moonshot Released 2026-01 Price — Length 262.1K | |||
| 14–21 | GPT-5.1 | OpenAI | Rating 19.8 ±3.4 | $1.25 | $10 | 400K | OpenAI Released 2025-11 Price $1.25 / $10 Length 400K | |||
| 15–23 | Gemini 2.5 Pro (preview, 2025-05) | Rating 17.7 ±3.3 | — | — | 1.05M | Google Released 2025-05 Price — Length 1.05M | ||||
| 17–23 | o3 | OpenAI | Rating 16.3 ±3.3 | $2 | $8 | 200K | OpenAI Released 2025-04 Price $2 / $8 Length 200K | |||
| 18–23 | GPT-5 mini | OpenAI | Rating 15.4 ±3.2 | $0.25 | $2 | 400K | OpenAI Released 2025-08 Price $0.25 / $2 Length 400K | |||
| 19–23 | Gemini 2.5 Pro (exp) | Rating 14.0 ±3.1 | — | — | 1.05M | Google Released 2025-03 Price — Length 1.05M | ||||
| 19–23 | o4-mini | OpenAI | Rating 13.9 ±3.1 | $1.1 | $4.4 | 200K | OpenAI Released 2025-04 Price $1.1 / $4.4 Length 200K | |||
| 24–28 | Claude Sonnet 4.5 | Anthropic | Rating 9.4 ±2.8 | $3 | $15 | 200K | Anthropic Released 2025-09 Price $3 / $15 Length 200K | |||
| 24–28 | Gemini 2.5 Flash (preview, 2025-04) | Rating 7.6 ±2.6 | — | — | 1.05M | Google Released 2025-04 Price — Length 1.05M | ||||
| 24–30 | Claude Opus 4.1 | Anthropic | Rating 7.1 ±2.6 | — | — | 200K | Anthropic Released 2025-08 Price — Length 200K | |||
| 24–34 | Gemini 2.5 Flash (preview, 2025-05) | Rating 6.5 ±2.5 | — | — | 1.05M | Google Released 2025-05 Price — Length 1.05M | ||||
| 24–35 | Claude Opus 4 | Anthropic | Rating 6.2 ±2.5 | — | — | 200K | Anthropic Released 2025-05 Price — Length 200K | |||
| 26–36 | Gemini 3.1 Flash-Lite | Rating 4.0 ±2.3 | $0.25 | $1.5 | 1.05M | Google Released 2026-03 Price $0.25 / $1.5 Length 1.05M | ||||
| 26–37 | GLM-4.5 | Z.ai | Rating 3.7 ±2.2 | $0.6 | $2.2 | 131.1K | Z.ai Released 2025-08 Price $0.6 / $2.2 Length 131.1K | |||
| 27–39 | GLM-4.5 Air | Z.ai | Rating 3.5 ±2.2 | $0.2 | $1.1 | 131.1K | Z.ai Released 2025-07 Price $0.2 / $1.1 Length 131.1K | |||
| 27–39 | o1-pro | OpenAI | Rating 3.5 ±2.2 | $150 | $600 | 200K | OpenAI Released 2025-03 Price $150 / $600 Length 200K | |||
| 27–39 | Claude 3.7 Sonnet | Anthropic | Rating 3.4 ±2.2 | — | — | 200K | Anthropic Released 2025-02 Price — Length 200K | |||
| 27–39 | o1 | OpenAI | Rating 3.3 ±2.2 | $15 | $60 | 128K | OpenAI Released 2024-12 Price $15 / $60 Length 128K | |||
| 28–39 | Claude Sonnet 4 | Anthropic | Rating 3.1 ±2.2 | — | — | 200K | Anthropic Released 2025-05 Price — Length 200K | |||
| 29–42 | Gemini 2.0 Flash (thinking, exp) | Rating 1.8 ±2.0 | — | — | — | Google Released 2024-12 Price — Length — | ||||
| 30–44 | Llama 4 Maverick | Meta | Rating 0.9 ±1.9 | — | — | 524.3K | Meta Released 2025-04 Price — Length 524.3K | |||
| 31–44 | GPT-4.5 (preview) | OpenAI | Rating 0.7 ±1.8 | — | — | — | OpenAI Released 2025-02 Price — Length — | |||
| 31–44 | GPT-4.1 | OpenAI | Rating 0.6 ±1.8 | $2 | $8 | 1.05M | OpenAI Released 2025-04 Price $2 / $8 Length 1.05M | |||
| 36–45 | Gemini 1.5 Pro 002 | Rating 0.0 ±1.7 | — | — | — | Google Released 2024-09 Price — Length — | ||||
| 36–45 | Mistral Medium 3 | Mistral | Rating 0.0 ±1.7 | $0.4 | $2 | 128K | Mistral Released 2025-05 Price $0.4 / $2 Length 128K | |||
| 36–45 | Amazon Nova Pro 1.0 | Amazon | Rating 0.0 ±1.6 | — | — | 300K | Amazon Released 2024-12 Price — Length 300K | |||
| 37–45 | Claude 3.5 Sonnet (2024-10) | Anthropic | Rating 0.0 ±1.6 | — | — | 200K | Anthropic Released 2024-10 Price — Length 200K | |||
| 37–45 | Amazon Nova Lite 1.0 | Amazon | Rating 0.0 ±1.5 | — | — | 300K | Amazon Released 2024-12 Price — Length 300K | |||
| 40–45 | GPT-4o (2024-11) | OpenAI | Rating 0.0 ±1.3 | $2.5 | $10 | 128K | OpenAI Released 2024-11 Price $2.5 / $10 Length 128K |
Best accuracy over time
2024-09-24Release date2026-10-07
How to read this page
- Some are multiple choice and some need a short written answer; the value is accuracy. Figures published by the organizations that made the tasks, among others, collected here.
- The accuracy is the share of problems solved, with the part you could get by guessing removed.
- In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
- Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
- Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.
Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-10-24, data sources