Task results
Translation from Japanese to English (WMT20jaen)
Has the model translate news sentences from Japanese to English and measures how close they are to human reference translations.
How to read this table
- Higher rows are rated higher. A range like “1–2” means the gap is within the margin of error.
- The score is the model's result on a shared set of problems, in points.
- Small gaps do not mean much.
- Prices are a guide per million tokens, reading then writing.
- Length is how much text the model can read at once (tokens).
| Rank | Model | Developer | Score | Evaluated | Japanese (human rating) | Input $ /1M | Output $ /1M | Context | Supports | Weights | Released | Price and specs |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-4o (2024-08) | OpenAI | Rating 26.5 | 1292rank 124–185 | $2.5 | $10 | 128K | OpenAI Released 2024-08 Price $2.5 / $10 Length 128K | ||||
| 2 | Gemma 4 26B A4B | Rating 26.1 | — | $0.15 | $0.6 | 262.1K | Google Released 2026-04 Price $0.15 / $0.6 Length 262.1K | |||||
| 3–4 | Gemma 4 31B | Rating 26.0 | — | — | — | 262.1K | Google Released 2026-04 Price — Length 262.1K | |||||
| 3–4 | GPT-4.1 | OpenAI | Rating 26.0 | 1347rank 83–150 | $2 | $8 | 1.05M | OpenAI Released 2025-04 Price $2 / $8 Length 1.05M | ||||
| 5 | Llama 3.3 Swallow 70B (instruct, v0.4) | Swallow (Science Tokyo) | Rating 25.1 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 6 | GPT-5.4 | OpenAI | Rating 24.9 | 1480rank 2–50 | $2.5 | $15 | 1.05M | OpenAI Released 2026-03 Price $2.5 / $15 Length 1.05M | ||||
| 7 | Qwen3.5 122B A10B | Alibaba | Rating 24.7 | 1361rank 60–154 | $0.4 | $3.2 | 262.1K | Alibaba Released 2026-02 Price $0.4 / $3.2 Length 262.1K | ||||
| 8 | Llama 3.3 70B (instruct) | Meta | Rating 24.6 | 1214rank 176–224 | — | — | 131.1K | Meta Released 2024-12 Price — Length 131.1K | ||||
| 9 | Qwen3.5 27B | Alibaba | Rating 24.1 | 1339rank 71–166 | $0.3 | $2.4 | 262.1K | Alibaba Released 2026-02 Price $0.3 / $2.4 Length 262.1K | ||||
| 10–11 | GPT-5 mini | OpenAI | Rating 23.9 | 1364rank 63–146 | $0.25 | $2 | 400K | OpenAI Released 2025-08 Price $0.25 / $2 Length 400K | ||||
| 10–11 | Qwen3.5 35B A3B | Alibaba | Rating 23.9 | 1315rank 93–183 | $0.25 | $2 | 262.1K | Alibaba Released 2026-02 Price $0.25 / $2 Length 262.1K | ||||
| 12 | Gemma 3 27B | Rating 23.8 | 1308rank 117–173 | — | — | 128K | Google Released 2025-03 Price — Length 128K | |||||
| 13 | GPT-5 | OpenAI | Rating 23.6 | 1431rank 17–93 | $1.25 | $10 | 400K | OpenAI Released 2025-08 Price $1.25 / $10 Length 400K | ||||
| 14–15 | Llama 4 Scout | Meta | Rating 23.0 | 1259rank 149–209 | — | — | 128K | Meta Released 2025-04 Price — Length 128K | ||||
| 14–15 | Qwen3 235B A22B (instruct) | Alibaba | Rating 23.0 | 1392rank 44–119 | — | — | 128K | Alibaba Released 2025-07 Price — Length 128K | ||||
| 16–17 | Gemma 3 12B | Rating 22.9 | — | — | — | 131K | Google Released 2025-03 Price — Length 131K | |||||
| 16–17 | Qwen3 235B A22B (thinking) | Alibaba | Rating 22.9 | 1376rank 49–142 | — | — | 262.1K | Alibaba Released 2025-07 Price — Length 262.1K | ||||
| 18–21 | ELYZA-Shortcut-1.0-Qwen-32B | ELYZA | Rating 22.8 | — | — | — | — | — | — | ELYZA Released — Price — Length — | ||
| 18–21 | Gemma 4 E4B | Rating 22.8 | — | — | — | 131.1K | — | — | Google Released — Price — Length 131.1K | |||
| 18–21 | Qwen3 Next 80B A3B (instruct) | Alibaba | Rating 22.8 | 1369rank 53–146 | $0.5 | $2 | 129K | Alibaba Released 2025-09 Price $0.5 / $2 Length 129K | ||||
| 18–21 | Qwen3-Swallow-32B-CPT-v0.2 | Swallow (Science Tokyo) | Rating 22.8 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 22 | Gemma 4 E2B | Rating 22.7 | — | — | — | 131.1K | — | — | Google Released — Price — Length 131.1K | |||
| 23–25 | Llama 3.1 Swallow 8B (instruct, v0.5) | Swallow (Science Tokyo) | Rating 22.2 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 23–25 | NVIDIA Nemotron 3 Super 120B A12B | NVIDIA | Rating 22.2 | — | — | — | 262.1K | NVIDIA Released 2026-03 Price — Length 262.1K | ||||
| 23–25 | Qwen3-Swallow-30B-A3B-CPT-v0.2 | Swallow (Science Tokyo) | Rating 22.2 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 26 | o3-mini | OpenAI | Rating 22.1 | 1315rank 101–175 | $1.1 | $4.4 | 200K | OpenAI Released 2025-01 Price $1.1 / $4.4 Length 200K | ||||
| 27 | Qwen3 14B | Alibaba | Rating 21.9 | — | $0.35 | $1.4 | 131.1K | Alibaba Released 2025-04 Price $0.35 / $1.4 Length 131.1K | ||||
| 28 | GPT-OSS-Swallow-120B-RL-v0.1 | Swallow (Science Tokyo) | Rating 21.8 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 29–32 | ELYZA-Thinking-1.0-Qwen-32B | ELYZA | Rating 21.6 | — | — | — | — | — | — | ELYZA Released — Price — Length — | ||
| 29–32 | GPT-OSS-Swallow-20B-SFT-v0.1 | Swallow (Science Tokyo) | Rating 21.6 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 29–32 | o3 | OpenAI | Rating 21.6 | 1424rank 22–95 | $2 | $8 | 200K | OpenAI Released 2025-04 Price $2 / $8 Length 200K | ||||
| 29–32 | Qwen3 32B | Alibaba | Rating 21.6 | — | $0.7 | $2.8 | 131.1K | Alibaba Released 2025-04 Price $0.7 / $2.8 Length 131.1K | ||||
| 33–34 | Qwen3 30B A3B | Alibaba | Rating 21.5 | 1254rank 150–211 | — | — | 128K | Alibaba Released 2025-04 Price — Length 128K | ||||
| 33–34 | Qwen3-Swallow-32B-RL-v0.2 | Swallow (Science Tokyo) | Rating 21.5 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 35 | GPT-OSS-Swallow-120B-SFT-v0.1 | Swallow (Science Tokyo) | Rating 21.3 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 36–38 | ABEJA-QwQ32b-Reasoning-Japanese-v1.0 | ABEJA | Rating 21.0 | — | — | — | — | — | — | ABEJA Released — Price — Length — | ||
| 36–38 | GPT-OSS-Swallow-20B-RL-v0.1 | Swallow (Science Tokyo) | Rating 21.0 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 36–38 | Qwen3-Swallow-32B-SFT-v0.2 | Swallow (Science Tokyo) | Rating 21.0 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 39–40 | Qwen3.5 9B | Alibaba | Rating 20.9 | — | — | — | 262K | Alibaba Released 2026-02 Price — Length 262K | ||||
| 39–40 | Qwen3-Swallow-30B-A3B-RL-v0.2 | Swallow (Science Tokyo) | Rating 20.9 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 41 | GPT-OSS 120B | OpenAI | Rating 20.8 | 1325rank 95–166 | — | — | 128K | OpenAI Released 2025-08 Price — Length 128K | ||||
| 42–43 | Qwen3-Swallow-30B-A3B-SFT-v0.2 | Swallow (Science Tokyo) | Rating 20.7 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 42–43 | QwQ Bakeneko 32B | rinna | Rating 20.7 | — | — | — | — | — | — | rinna Released — Price — Length — | ||
| 44 | Qwen3-Swallow-8B-CPT-v0.2 | Swallow (Science Tokyo) | Rating 20.6 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 45 | GPT-OSS 20B | OpenAI | Rating 20.5 | 1256rank 140–216 | — | — | 131K | OpenAI Released 2025-08 Price — Length 131K | ||||
| 46–47 | Qwen3 8B | Alibaba | Rating 20.1 | — | $0.18 | $0.7 | 131.1K | Alibaba Released 2025-04 Price $0.18 / $0.7 Length 131.1K | ||||
| 46–47 | Qwen3-Swallow-8B-RL-v0.2 | Swallow (Science Tokyo) | Rating 20.1 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 48 | Qwen3-Swallow-8B-SFT-v0.2 | Swallow (Science Tokyo) | Rating 19.9 | — | — | — | — | — | — | Swallow (Science Tokyo) Released — Price — Length — | ||
| 49 | NVIDIA-Nemotron-Nano-9B-v2-Japanese | NVIDIA | Rating 19.6 | — | — | — | — | — | — | NVIDIA Released — Price — Length — | ||
| 50 | Qwen3 Next 80B A3B | Alibaba | Rating 19.5 | 1293rank 112–202 | $0.5 | $6 | 131.1K | Alibaba Released 2025-09 Price $0.5 / $6 Length 131.1K | ||||
| 51 | Llama 3.1 8B (instruct) | Meta | Rating 19.4 | 1087rank 232–251 | — | — | 128K | Meta Released 2024-07 Price — Length 128K | ||||
| 52 | DeepSeek-R1-Distill-Qwen-32B-Japanese | CyberAgent | Rating 19.1 | — | — | — | — | — | — | CyberAgent Released — Price — Length — | ||
| 53 | LLM-jp-4 32B A3B | LLM-jp | Rating 19.0 | — | — | — | — | — | — | LLM-jp Released — Price — Length — | ||
| 54–56 | Gemma 3 4B | Rating 18.9 | — | — | — | 131K | Google Released 2025-03 Price — Length 131K | |||||
| 54–56 | LLM-jp-4 8B | LLM-jp | Rating 18.9 | — | — | — | — | — | — | LLM-jp Released — Price — Length — | ||
| 54–56 | Qwen3 4B | Alibaba | Rating 18.9 | — | — | — | — | Alibaba Released 2025-04 Price — Length — | ||||
| 57 | OLMo 3 32B Think | Ai2 | Rating 18.5 | — | — | — | — | — | Ai2 Released — Price — Length — | |||
| 58 | LLM-jp-3.1 13B (instruct4) | LLM-jp | Rating 16.1 | — | — | — | — | — | — | LLM-jp Released — Price — Length — | ||
| 59 | Sarashina2.2 3B (instruct, v0.1) | SB Intuitions | Rating 16.0 | — | — | — | — | — | — | SB Intuitions Released — Price — Length — | ||
| 60 | LLM-jp-3.1 1.8B (instruct4) | LLM-jp | Rating 15.9 | — | — | — | — | — | — | LLM-jp Released — Price — Length — | ||
| 61 | Qwen3 1.7B | Alibaba | Rating 15.1 | — | — | — | — | Alibaba Released 2025-04 Price — Length — | ||||
| 62 | Qwen3.5 4B | Alibaba | Rating 14.7 | — | — | — | 262.1K | Alibaba Released 2026-02 Price — Length 262.1K | ||||
| 63 | OLMo 3 7B Think | Ai2 | Rating 14.1 | — | — | — | — | — | — | Ai2 Released — Price — Length — | ||
| 64 | Gemma 3 1B | Rating 8.3 | — | — | — | — | Google Released 2025-03 Price — Length — | |||||
| 65 | Llama 3.1 Nemotron Nano 8B (v1) | NVIDIA | Rating 6.9 | — | — | — | — | — | — | NVIDIA Released — Price — Length — | ||
| 66 | Qwen3.5 2B | Alibaba | Rating 0.2 | — | — | — | 32.8K | Alibaba Released 2026-02 Price — Length 32.8K | ||||
| 67–68 | Qwen3 0.6B | Alibaba | Rating 0.0 | — | — | — | — | Alibaba Released 2025-04 Price — Length — | ||||
| 67–68 | Qwen3.5 0.8B | Alibaba | Rating 0.0 | — | — | — | 32.8K | Alibaba Released 2026-02 Price — Length 32.8K |
How to read this page
- The values are closeness to reference translations on a 100-point scale, and even the top models score around 25 to 30. Differences under 1 point are not clear differences.
- The human rating comes from people comparing two answers and picking the better one (higher is better).
- In the Supports column the icons are, from left: can use tools / can read images / can answer in a fixed format (JSON); faded means not supported or unknown.
- Open-weights models have their contents (the weights) published, so you can also run them on your own computer or server. Models without the open-weights mark are normally used through the developer's own services (apps or API); for some models the weights status is unknown.
- Names such as ChatGPT next to a model are the consumer services its developer offers. Check on the service itself whether that model is available there.
Data: Arena, Epoch AI, Swallow LLM team (CC BY 4.0), Models.dev (MIT), updated 2026-10-24, data sources