ExamenosMellum2.1 Thinking
Explore Mellum2.1 Thinking from JetBrains: published specifications, source-linked vendor benchmarks, independent evaluator coverage and recorded pricing when available.
Compare Mellum2.1 Thinking with other models →Compare subscription plansExplore data coverage
Pricing · OpenRouter
No OpenRouter pricing is recorded for this model. Missing rates are not free.
Official / vendor benchmarks
Default headline records. Own-vendor, peer-vendor and third-party provenance remain visible in evidence; configurations may differ.
| Benchmark / evaluator | Headline score | Evidence |
|---|---|---|
| LiveCodeBench v6 | 82%unknown effort | All 1 recorded result & sources82% · raw 82 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| HumanEval+ | 91.5%unknown effort | All 1 recorded result & sources91.5% · raw 91.5 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| MBPP+ | 79.4%unknown effort | All 1 recorded result & sources79.4% · raw 79.4 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| AIME 2025–2026 mean (JetBrains) | 83.3%unknown effort | All 1 recorded result & sources83.3% · raw 83.3 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| GSM-Plus | 88.3%unknown effort | All 1 recorded result & sources88.3% · raw 88.3 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| SWE-bench Verified | 47%unknown effort | All 1 recorded result & sources47% · raw 47 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode; Pi v0.73.1 agent harness (shell and file tools), each model default sampling, 114K-token context, up to 16K tokens per turn. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| Terminal-Bench 2.1 | 17.4%unknown effort | All 1 recorded result & sources17.4% · raw 17.4 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode; Pi v0.73.1 agent harness (shell and file tools), each model default sampling, 114K-token context, up to 16K tokens per turn. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| SWE-bench Pro (revision unspecified) | 28%unknown effort | All 1 recorded result & sources28% · raw 28 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode; Pi v0.73.1 agent harness (shell and file tools), each model default sampling, 114K-token context, up to 16K tokens per turn. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| BFCL v4 macro-average (JetBrains) | 62.3%unknown effort | All 1 recorded result & sources62.3% · raw 62.3 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| WorkBench | 44.6%unknown effort | All 1 recorded result & sources44.6% · raw 44.6 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| ToolHop | 49.1%unknown effort | All 1 recorded result & sources49.1% · raw 49.1 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| IFEval | 90.6%unknown effort | All 1 recorded result & sources90.6% · raw 90.6 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| GPQA Diamond | 64.6%unknown effort | All 1 recorded result & sources64.6% · raw 64.6 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| MMLU-Redux | 87.8%unknown effort | All 1 recorded result & sources87.8% · raw 87.8 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| MixEval-Hard | 46.4%unknown effort | All 1 recorded result & sources46.4% · raw 46.4 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| XSTest | 88.8%unknown effort | All 1 recorded result & sources88.8% · raw 88.8 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
| HarmBench (lower is better) | 8.5%unknown effort | All 1 recorded result & sources8.5% · raw 8.5 % Headline · unknown effort · Own vendor Source/record date: 2026-10-08 Harness: JetBrains pipeline, thinking mode, greedy decoding. JetBrains self-reported on the released Mellum2.1 Thinking checkpoint; evaluation date unpublished; thinking mode has no effort tier. https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking |
Independent evaluators
Evaluator harnesses are distinct from vendor measurements. Missing coverage is not a failed test.
No independent evaluator results are recorded for this model.
Read how we select and source scores or the comparison guide.