Data Sources and License

Data used

From Epoch AI's Benchmark data, the benchmarks Epoch AI ran itself and 12 tables in which Epoch AI collected figures reported by other organizations are used.

Tables of figures reported by others (type "Reported by others" in the list) use the values as in the source data. When the same model was run in several ways (agent setups or settings), the row with the best value is kept. ± is shown only for tables with an error column (a 95% confidence interval column is divided by 1.96 to give the standard error).

From Arena's Leaderboard dataset (Hugging Face), the latest version of the text conversation category with the effect of response length and formatting removed is used.

Japanese and English tasks: per-task values (0 to 1) for instruction-tuned models from the Swallow LLM Leaderboard (Swallow LLM team, CC BY 4.0), multiplied by 100 and shown as % (translation (WMT20) is shown as points, not %). The source data has no error, so no ± is shown, and only models with the same value share a rank range. The evaluation date is the date in the source data.

Price and context: values from Models.dev (MIT), attached only to the models listed here. Prices are the developer's own published prices (input / output, US dollars per 1M tokens); prices from other hosting providers are not used. Context is shown with K for 1,000 tokens and M for 1M tokens. The release date is Epoch AI's value; for models without one, it is the earliest date in the developer's own records from the other sources (excluding January 1, which a source sets when only the year is known).

Yen prices on the Japanese pages are converted from US dollars with the European Central Bank reference rate (via Frankfurter, fetched weekly). English pages keep US dollars.

Developer service names come from a fixed mapping table, not from the imported data.

Support marks (Tools, Image, JSON): Models.dev's tool_call, image in modalities.input and structured_output. Models without a value are "unknown" and are left out when you filter on them.

License and retrieval

Epoch AI, Arena and the Swallow LLM Leaderboard are published under CC BY 4.0, and Models.dev under MIT. The data is imported once a week by fixed rules. A benchmark or category that disappears from the source data, or source data whose format or row count is off, keeps its previous values and its page shows the date updates stopped.

• Arena, Leaderboard dataset (CC BY 4.0). Updated 2026-⁠10-⁠24
• Epoch AI, Benchmark data (CC BY 4.0). Updated 2026-⁠10-⁠24
• Swallow LLM team, Swallow LLM Leaderboard (CC BY 4.0). Updated 2026-⁠10-⁠14
• Models.dev (MIT). Updated 2026-⁠10-⁠24
• Frankfurter, European Central Bank reference rates. Updated 2026-⁠10-⁠23

How the numbers are calculated

Arena human ratings are rounded to integers and shown with half of the interval (lower to upper bound) as ±. Models whose intervals overlap share a rank range: the top end is 1 + the number of models whose lower bound is above this model's upper bound, and the bottom end is the number of models whose upper bound is at or above this model's lower bound.

The Epoch AI score is mean_score, which comes with a standard error (stderr). Rows without mean_score use Best score and show no error. Two models fall in the same rank range when their difference is within 1.96 × √(se1² + se2²). Rows without stderr are not grouped with others and are ranked by value alone.

Task value = (score − random_baseline) ÷ (score_ceiling − random_baseline), shown from 0 to 100 (the part a random guess would get is removed). Depending on the task it is called accuracy, success rate or score. It is not shown for benchmarks without random_baseline and score_ceiling. The ± in the tables and bars is 1.96 × the standard error (the 95% range), converted to the task value scale. Values and ± are rounded to one decimal place.

Weights: open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown. Models whose Arena license is not proprietary are shown as "Open" (models with published weights that you can run on your own servers), and others as "Closed". Models without a license use Epoch AI's accessibility group, or else the open-weights flag (open_weights) in the developer's own record on Models.dev; if neither is available, the weights column shows "—".

Week change: the human rating difference (integer) from the newest history snapshot kept at each import that is at least 6 days older than the latest one. Models not in that snapshot are marked "New".

There are two price axes. Standard = (3 × input + output) ÷ 4 (a 3 : 1 weighted average of input and output, an industry convention). Input-heavy = the input price (for uses where input tokens make up most of the cost). Both are US dollars per 1M tokens.

Best at each price (the best-rated model at each price): going from cheapest to most expensive on the use case's price axis, every model rated higher on the ranking rating than every cheaper model is listed (at the same price, the higher-rated one), and the table shows them best first. So within the price you can pay, no model is rated higher than the top row that fits. The first row is the highest-rated model with a known price. A row is marked △ when its gain over the nearest row below it without △ is within the sum of their ± ranges (the ranges overlap); the last row and rows where either has no ± range are not marked. A △ row costs more than the nearest row below it without △, yet its rating is not clearly higher, so picking the top row without △ within the price you can pay is enough. Among rows with ± ranges, a row without △ is clearly rated higher than the nearest row below it without △. Choosing a price limit does not rebuild the table; it only hides the rows priced above the limit (the △ marks stay the same). Models without a known price are left out.

The ● in the chart and the mark in the all-models table: only the models listed in the best-at-each-price table (including △ rows). No score mixing ratings and price, and no score mixing the reference ratings, is made.

Models that differ only in reasoning settings (high, max and so on) are combined into one row using the setting with the highest value on that page. The value for each setting is shown in the compare table (on a PC, also when you hover over the model name). Entries that differ only in the hosting provider (another company serving the same model) are also treated as the same model, using the largest context length. The release date is Epoch AI's value; for models without one, it is the earliest date in the developer's own records from the other sources (excluding January 1, which a source sets when only the year is known). Different sizes (mini and so on) are separate models. Dated versions are covered under "Matching model names".

No combined score (a score mixing several ratings) across benchmarks is made.

Matching model names

Model names are normalized by lowercasing, turning spaces, _ and . into -, and removing parentheses. An Arena name and an Epoch AI Model version or display name that normalize to the same string are treated as one model; others are kept as separate models per source. Names that become the same once their spelling is aligned are also combined. When two or more versions fall in the same month, their names keep the day; names with different dates are treated as different versions. A name without a date is kept as a single version. However, when a name without a date and a name with a date form a single pair and the date matches the release month of the undated one, they are one model in one row. Names shown on the pages turn dates in API IDs and English month names into year and month (year, month and day when two or more fall in the same month) and follow the official spelling, with instruct, version, quantization, year and month, and the extra words at the end of the name (such as preview or thinking) grouped in parentheses. Names whose release date is known and that do not need the date to tell versions apart drop the year and month (the release date column shows it). A version that keeps a date in its name shows a release date only when that version's own date is known; the original model's date is not used.