Task results
Translation from English to Japanese (WMT20enja)
Has the model translate news sentences from English to Japanese and measures how close they are to human reference translations.
How to read this table
- Higher rows are rated higher. A range like “1–2” means the gap is within the margin of error.
- The score is the model's result on a shared set of problems, in points.
- Small gaps do not mean much.
- Prices are a guide per million tokens, reading then writing.
- Length is how much text the model can read at once (tokens).
| Rank | Model | Developer | Score | Evaluated | Japanese (human rating) | Input $ /1M | Output $ /1M | Context | Supports | Weights | Released | Price and specs |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Gemma 4 31B | Rating 28.5 | — | — | — | 262.1K | Google Released 2026-04 Price — Length 262.1K | |||||
| 2 | GPT-4o (2024-08) | OpenAI | Rating 28.2 | 1292rank 124–185 | $2.5 | $10 | 128K | OpenAI Released 2024-08 Price $2.5 / $10 Length 128K | ||||
| 3 | Gemma 4 26B A4B | Rating 28.0 | — | $0.15 | $0.6 | 262.1K | Google Released 2026-04 Price $0.15 / $0.6 Length 262.1K | |||||
| 4 | GPT-4.1 | OpenAI | Rating 27.8 | 1347rank 83–150 | $2 | $8 | 1.05M | OpenAI Released 2025-04 Price $2 / $8 Length 1.05M | ||||
| 5 | o3 | OpenAI | Rating 27.6 | 1424rank 22–95 | $2 | $8 | 200K | OpenAI Released 2025-04 Price $2 / $8 Length 200K | ||||
| 6 | Llama 3.3 Swallow 70B (instruct, v0.4) | Swallow (Science Tokyo) | Rating 27.5 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 7–8 | GPT-5.4 | OpenAI | Rating 27.2 | 1480rank 2–50 | $2.5 | $15 | 1.05M | OpenAI Released 2026-03 Price $2.5 / $15 Length 1.05M | ||||
| 7–8 | GPT-5 | OpenAI | Rating 27.2 | 1431rank 17–93 | $1.25 | $10 | 400K | OpenAI Released 2025-08 Price $1.25 / $10 Length 400K | ||||
| 9 | GPT-5 mini | OpenAI | Rating 26.7 | 1364rank 63–146 | $0.25 | $2 | 400K | OpenAI Released 2025-08 Price $0.25 / $2 Length 400K | ||||
| 10 | GPT-OSS-Swallow-120B-SFT-v0.1 | Swallow (Science Tokyo) | Rating 26.4 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 11–12 | GPT-OSS-Swallow-120B-RL-v0.1 | Swallow (Science Tokyo) | Rating 26.3 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 11–12 | Qwen3.5 122B A10B | Alibaba | Rating 26.3 | 1361rank 60–154 | $0.4 | $3.2 | 262.1K | Alibaba Released 2026-02 Price $0.4 / $3.2 Length 262.1K | ||||
| 13 | GPT-OSS 120B | OpenAI | Rating 26.2 | 1325rank 95–166 | — | — | 128K | OpenAI Released 2025-08 Price — Length 128K | ||||
| 14 | Gemma 4 E4B | Rating 26.1 | — | — | — | 131.1K | — | — | Google Released — Price — Length 131.1K | |||
| 15 | Qwen3 235B A22B (instruct) | Alibaba | Rating 25.8 | 1392rank 44–119 | — | — | 128K | Alibaba Released 2025-07 Price — Length 128K | ||||
| 16 | Gemma 4 E2B | Rating 25.6 | — | — | — | 131.1K | — | — | Google Released — Price — Length 131.1K | |||
| 17 | Qwen3 235B A22B (thinking) | Alibaba | Rating 25.5 | 1376rank 49–142 | — | — | 262.1K | Alibaba Released 2025-07 Price — Length 262.1K | ||||
| 18 | Qwen3.5 27B | Alibaba | Rating 25.2 | 1339rank 71–166 | $0.3 | $2.4 | 262.1K | Alibaba Released 2026-02 Price $0.3 / $2.4 Length 262.1K | ||||
| 19–20 | Gemma 3 27B | Rating 25.0 | 1308rank 117–173 | — | — | 128K | Google Released 2025-03 Price — Length 128K | |||||
| 19–20 | Qwen3.5 35B A3B | Alibaba | Rating 25.0 | 1315rank 93–183 | $0.25 | $2 | 262.1K | Alibaba Released 2026-02 Price $0.25 / $2 Length 262.1K | ||||
| 21 | Llama 3.1 Swallow 8B (instruct, v0.5) | Swallow (Science Tokyo) | Rating 24.9 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 22 | GPT-OSS-Swallow-20B-RL-v0.1 | Swallow (Science Tokyo) | Rating 24.8 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 23–24 | Qwen3-Swallow-32B-RL-v0.2 | Swallow (Science Tokyo) | Rating 24.7 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 23–24 | Qwen3-Swallow-32B-SFT-v0.2 | Swallow (Science Tokyo) | Rating 24.7 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 25 | GPT-OSS-Swallow-20B-SFT-v0.1 | Swallow (Science Tokyo) | Rating 24.6 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 26–27 | ELYZA-Shortcut-1.0-Qwen-32B | ELYZA | Rating 24.5 | — | — | — | — | — | — | ELYZA Released — Price — Length — | ||
| 26–27 | Llama 3.3 70B (instruct) | Meta | Rating 24.5 | 1214rank 176–224 | — | — | 131.1K | Meta Released 2024-12 Price — Length 131.1K | ||||
| 28–29 | NVIDIA Nemotron 3 Super 120B A12B | NVIDIA | Rating 24.4 | — | — | — | 262.1K | NVIDIA Released 2026-03 Price — Length 262.1K | ||||
| 28–29 | Qwen3-Swallow-32B-CPT-v0.2 | Swallow (Science Tokyo) | Rating 24.4 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 30 | o3-mini | OpenAI | Rating 24.3 | 1315rank 101–175 | $1.1 | $4.4 | 200K | OpenAI Released 2025-01 Price $1.1 / $4.4 Length 200K | ||||
| 31–34 | Qwen3 Next 80B A3B (instruct) | Alibaba | Rating 24.0 | 1369rank 53–146 | $0.5 | $2 | 129K | Alibaba Released 2025-09 Price $0.5 / $2 Length 129K | ||||
| 31–34 | Qwen3 Next 80B A3B | Alibaba | Rating 24.0 | 1293rank 112–202 | $0.5 | $6 | 131.1K | Alibaba Released 2025-09 Price $0.5 / $6 Length 131.1K | ||||
| 31–34 | Qwen3-Swallow-30B-A3B-CPT-v0.2 | Swallow (Science Tokyo) | Rating 24.0 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 31–34 | Qwen3-Swallow-30B-A3B-SFT-v0.2 | Swallow (Science Tokyo) | Rating 24.0 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 35 | Qwen3-Swallow-30B-A3B-RL-v0.2 | Swallow (Science Tokyo) | Rating 23.9 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 36 | ABEJA-QwQ32b-Reasoning-Japanese-v1.0 | ABEJA | Rating 23.8 | — | — | — | — | — | — | ABEJA Released — Price — Length — | ||
| 37–38 | ELYZA-Thinking-1.0-Qwen-32B | ELYZA | Rating 23.7 | — | — | — | — | — | — | ELYZA Released — Price — Length — | ||
| 37–38 | Llama 4 Scout | Meta | Rating 23.7 | 1259rank 149–209 | — | — | 128K | Meta Released 2025-04 Price — Length 128K | ||||
| 39 | LLM-jp-4 32B A3B | LLM-jp | Rating 23.5 | — | — | — | — | — | — | LLM-jp Released — Price — Length — | ||
| 40 | GPT-OSS 20B | OpenAI | Rating 23.4 | 1256rank 140–216 | — | — | 131K | OpenAI Released 2025-08 Price — Length 131K | ||||
| 41 | QwQ Bakeneko 32B | rinna | Rating 23.2 | — | — | — | — | — | — | rinna Released — Price — Length — | ||
| 42 | Qwen3-Swallow-8B-CPT-v0.2 | Swallow (Science Tokyo) | Rating 23.0 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 43–44 | Qwen3 14B | Alibaba | Rating 22.8 | — | $0.35 | $1.4 | 131.1K | Alibaba Released 2025-04 Price $0.35 / $1.4 Length 131.1K | ||||
| 43–44 | Qwen3-Swallow-8B-SFT-v0.2 | Swallow (Science Tokyo) | Rating 22.8 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 45 | LLM-jp-4 8B | LLM-jp | Rating 22.7 | — | — | — | — | — | — | LLM-jp Released — Price — Length — | ||
| 46 | Gemma 3 12B | Rating 22.5 | — | — | — | 131K | Google Released 2025-03 Price — Length 131K | |||||
| 47 | Qwen3-Swallow-8B-RL-v0.2 | Swallow (Science Tokyo) | Rating 22.2 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 48 | Qwen3 32B | Alibaba | Rating 21.8 | — | $0.7 | $2.8 | 131.1K | Alibaba Released 2025-04 Price $0.7 / $2.8 Length 131.1K | ||||
| 49 | NVIDIA-Nemotron-Nano-9B-v2-Japanese | NVIDIA | Rating 20.7 | — | — | — | — | — | — | NVIDIA Released — Price — Length — | ||
| 50 | DeepSeek-R1-Distill-Qwen-32B-Japanese | CyberAgent | Rating 20.5 | — | — | — | — | — | — | CyberAgent Released — Price — Length — | ||
| 51 | Qwen3 8B | Alibaba | Rating 20.2 | — | $0.18 | $0.7 | 131.1K | Alibaba Released 2025-04 Price $0.18 / $0.7 Length 131.1K | ||||
| 52 | Llama 3.1 8B (instruct) | Meta | Rating 18.7 | 1087rank 232–251 | — | — | 128K | Meta Released 2024-07 Price — Length 128K | ||||
| 53 | Gemma 3 4B | Rating 18.6 | — | — | — | 131K | Google Released 2025-03 Price — Length 131K | |||||
| 54 | Qwen3 30B A3B | Alibaba | Rating 17.7 | 1254rank 150–211 | — | — | 128K | Alibaba Released 2025-04 Price — Length 128K | ||||
| 55 | Qwen3.5 9B | Alibaba | Rating 17.4 | — | — | — | 262K | Alibaba Released 2026-02 Price — Length 262K | ||||
| 56 | OLMo 3 32B Think | Ai2 | Rating 17.2 | — | — | — | — | — | Ai2 Released — Price — Length — | |||
| 57 | Qwen3 4B | Alibaba | Rating 14.9 | — | — | — | — | Alibaba Released 2025-04 Price — Length — | ||||
| 58–59 | OLMo 3 7B Think | Ai2 | Rating 12.5 | — | — | — | — | — | — | Ai2 Released — Price — Length — | ||
| 58–59 | Qwen3 1.7B | Alibaba | Rating 12.5 | — | — | — | — | Alibaba Released 2025-04 Price — Length — | ||||
| 60 | Qwen3.5 4B | Alibaba | Rating 8.0 | — | — | — | 262.1K | Alibaba Released 2026-02 Price — Length 262.1K | ||||
| 61 | Llama 3.1 Nemotron Nano 8B (v1) | NVIDIA | Rating 5.1 | — | — | — | — | — | — | NVIDIA Released — Price — Length — | ||
| 62 | LLM-jp-3.1 13B (instruct4) | LLM-jp | Rating 0.7 | — | — | — | — | — | — | LLM-jp Released — Price — Length — | ||
| 63 | Gemma 3 1B | Rating 0.4 | — | — | — | — | Google Released 2025-03 Price — Length — | |||||
| 64 | Sarashina2.2 3B (instruct, v0.1) | SB Intuitions | Rating 0.2 | — | — | — | — | — | — | SB Intuitions Released — Price — Length — | ||
| 65 | Qwen3 0.6B | Alibaba | Rating 0.1 | — | — | — | — | Alibaba Released 2025-04 Price — Length — | ||||
| 66–68 | LLM-jp-3.1 1.8B (instruct4) | LLM-jp | Rating 0.0 | — | — | — | — | — | — | LLM-jp Released — Price — Length — | ||
| 66–68 | Qwen3.5 0.8B | Alibaba | Rating 0.0 | — | — | — | 32.8K | Alibaba Released 2026-02 Price — Length 32.8K | ||||
| 66–68 | Qwen3.5 2B | Alibaba | Rating 0.0 | — | — | — | 32.8K | Alibaba Released 2026-02 Price — Length 32.8K |
How to read this page
- The values are closeness to reference translations on a 100-point scale, and even the top models score around 30. Differences under 1 point are not clear differences.
- The human rating comes from people comparing two answers and picking the better one (higher is better).
- In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
- Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
- Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.
Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-10-24, data sources