Skip to main content
Bench

Language

English日本語中文Español

Site info

Task results

Answering short factual questions correctly (SimpleQA Verified)

Short factual questions on politics, science and technology, art, sports, geography, music, and more, answered without answer choices.

How to read this table

  • Higher rows are rated higher. A range like “1–2” means the gap is within the margin of error.
  • The score is the share of a shared set of problems the model solved (%).
  • A gap of 3 means 3 problems out of 100, and ± is how far the score moves when re-measured.
  • Prices are a guide per million tokens, reading then writing.
  • Length is how much text the model can read at once (tokens).

± range of the top model

RankModelDeveloperAccuracy
(%)Accuracy scale 40 to 80		EvaluatedInput $
/1M		Output $
/1M		ContextSupportsWeightsReleasedPrice and specs
1–5Gemini 4 ArgonGoogleRating 76.0 ±2.6$2$101.05MGoogle Released 2026-⁠09 Price $2 / $10 Length 1.05M
1–5GPT-6 AstraOpenAIRating 75.6 ±2.7$10$501.05MOpenAI Released 2026-⁠09 Price $10 / $50 Length 1.05M
1–7GPT-6.1 SolOpenAIRating 73.9 ±2.7$2$101.05MOpenAI Released 2026-⁠09 Price $2 / $10 Length 1.05M
1–9Gemini 3.1 Pro (preview)GoogleRating 73.5 ±2.7$2$121.05MGoogle Released 2026-⁠02 Price $2 / $12 Length 1.05M
1–10Claude Opus 5.5AnthropicRating 72.2 ±2.8$4$201MAnthropic Released 2026-⁠09 Price $4 / $20 Length 1M
3–11Claude Fable 5.1AnthropicRating 70.8 ±2.8$10$501MAnthropic Released 2026-⁠09 Price $10 / $50 Length 1M
3–11Claude Fable 5AnthropicRating 70.7 ±2.8$10$501MAnthropic Released 2026-⁠06 Price $10 / $50 Length 1M
4–13Gemini 3.8 FlashGoogleRating 69.7 ±2.8$0.75$3.751.05MGoogle Released 2026-⁠09 Price $0.75 / $3.75 Length 1.05M
4–13GPT-5.6 SolOpenAIRating 69.7 ±2.8$4$201.05MOpenAI Released 2026-⁠07 Price $4 / $20 Length 1.05M
5–13Gemini 3.7 FlashGoogleRating 69.2 ±2.9$0.75$3.751.05MGoogle Released 2026-⁠08 Price $0.75 / $3.75 Length 1.05M
6–14Gemini 3 Flash (preview)GoogleRating 66.8 ±2.9$0.5$31MGoogle Released 2025-⁠12 Price $0.5 / $3 Length 1M
8–14Gemini 3.5 FlashGoogleRating 66.2 ±2.9$1.5$91.05MGoogle Released 2026-⁠05 Price $1.5 / $9 Length 1.05M
8–14Gemini 3.6 FlashGoogleRating 66.2 ±2.9$0.75$3.751.05MGoogle Released 2026-⁠07 Price $0.75 / $3.75 Length 1.05M
11–17GPT-5.5OpenAIRating 63.0 ±3.0$5$301.05MOpenAI Released 2026-⁠04 Price $5 / $30 Length 1.05M
14–19GPT-6 SolOpenAIRating 60.7 ±3.0$2$101.05MOpenAI Released 2026-⁠09 Price $2 / $10 Length 1.05M
14–20Muse Spark 1.2MetaRating 60.3 ±3.0$1.25$4.251.05MMeta Released 2026-⁠08 Price $1.25 / $4.25 Length 1.05M
14–21Claude Opus 5AnthropicRating 59.9 ±3.0$5$251MAnthropic Released 2026-⁠07 Price $5 / $25 Length 1M
15–21Muse Spark 1.1MetaRating 57.8 ±3.1$1.25$4.251MMeta Released 2026-⁠07 Price $1.25 / $4.25 Length 1M
15–23Muse Spark Open⁠MetaRating 56.6 ±3.1——1.04MMeta Released 2026-⁠10 Price — Length 1.04M
16–25Grok 4.7xAIRating 56.0 ±3.1$2$6500KxAI Released 2026-⁠09 Price $2 / $6 Length 500K
17–25Qwen3.7 MaxAlibabaRating 55.8 ±3.1$2.5$7.51MAlibaba Released 2026-⁠05 Price $2.5 / $7.5 Length 1M
19–30Claude Opus 4.8AnthropicRating 53.0 ±3.1$5$251MAnthropic Released 2026-⁠05 Price $5 / $25 Length 1M
19–30DeepSeek-V4 Pro (2026-⁠08)⁠DeepSeekRating 52.9 ±3.1$0.66$1.981MDeepSeek Released 2026-⁠08 Price $0.66 / $1.98 Length 1M
20–32Qwen3.6 Max (preview)AlibabaRating 52.0 ±3.1$1.3$7.8240KAlibaba Released 2026-⁠04 Price $1.3 / $7.8 Length 240K
20–32Claude Opus 4.7AnthropicRating 51.7 ±3.1$5$251MAnthropic Released 2026-⁠04 Price $5 / $25 Length 1M
22–37Kimi K3⁠MoonshotRating 50.6 ±3.1$3$151.05MMoonshot Released 2026-⁠07 Price $3 / $15 Length 1.05M
22–38GPT-5OpenAIRating 50.1 ±3.1$1.25$10400KOpenAI Released 2025-⁠08 Price $1.25 / $10 Length 400K
22–40o3OpenAIRating 49.4 ±3.1$2$8200KOpenAI Released 2025-⁠04 Price $2 / $8 Length 200K
22–40Grok 4.6xAIRating 49.3 ±3.1$2$6500KxAI Released 2026-⁠08 Price $2 / $6 Length 500K
22–40Qwen3 Max (instruct)AlibabaRating 48.7 ±3.1——258KAlibaba Released 2025-⁠09 Price — Length 258K
24–41Grok 4.5xAIRating 48.3 ±3.1$2$6500KxAI Released 2026-⁠07 Price $2 / $6 Length 500K
24–41GPT-5.1OpenAIRating 48.0 ±3.1$1.25$10400KOpenAI Released 2025-⁠11 Price $1.25 / $10 Length 400K
26–42Qwen3.8 Max (2026-⁠09)AlibabaRating 47.3 ±3.1——1MAlibaba Released 2026-⁠09 Price — Length 1M
26–43Claude Opus 4.6AnthropicRating 47.0 ±3.1$5$251MAnthropic Released 2026-⁠02 Price $5 / $25 Length 1M
26–43DeepSeek-V4 Pro⁠DeepSeekRating 47.0 ±3.1$0.66$1.981MDeepSeek Released 2026-⁠04 Price $0.66 / $1.98 Length 1M
26–44Claude Sonnet 5.5AnthropicRating 46.5 ±3.1$2$101MAnthropic Released 2026-⁠09 Price $2 / $10 Length 1M
26–44GPT-5.4 ProOpenAIRating 46.3 ±3.1$30$1801.05MOpenAI Released 2026-⁠03 Price $30 / $180 Length 1.05M
27–44Qwen3.8 MaxAlibabaRating 45.8 ±3.1$2$61MAlibaba Released 2026-⁠08 Price $2 / $6 Length 1M
28–45Claude Opus 4.5AnthropicRating 45.7 ±3.1$5$25200KAnthropic Released 2025-⁠11 Price $5 / $25 Length 200K
28–48GPT-5.4OpenAIRating 45.1 ±3.1$2.5$151.05MOpenAI Released 2026-⁠03 Price $2.5 / $15 Length 1.05M

Best accuracy over time

2024-02-29Release date2026-10-17
How to read this page
  • Google removed similar questions from the original SimpleQA and fixed topic imbalance and answer key errors. The value is accuracy.
  • The accuracy is the share of problems solved. A gap of 1 point is 1 question out of 100.
  • In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
  • Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
  • Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.

Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-⁠10-⁠24, data sources