Gemma 4 31B Benchmarks, Specifications & Availability

Gemma 4 31B

Explore Gemma 4 31B from Google: published specifications, source-linked vendor benchmarks, independent evaluator coverage and recorded pricing when available.

Published specifications

Provider: Google
Access: Open weights
License: Apache-2.0
Context window: 256K
Total parameters: 30.7B
Active parameters: Not published
Released: 2026-03-31
Modalities: text, image, video
Family: Gemma 4

Hugging Face · Announcement · Website · OpenRouter

Model notes

Public ungated Apache-2.0 instruction-tuned checkpoint; PT, assistant/MTP and QAT builds are not extra curated models. Vendor architecture size is retained separately from HF tensor totals including modality modules. Dense language backbone30.7B plus vision encoder; no active-parameter count inferred. No native audio input; video uses frames (up to60 seconds at1fps), text output only. Context is the published configured limit, not a universal guarantee of whole-window accuracy. Reasoning is an on/off mode, not an API effort level. Report Table9 disables thinking; vision uses1120 or280 visual-token budgets. OmniDocBench uses native average edit distance, not accuracy. Card/report audio aggregates and the older launch τ² figures differ and remain source-specific observations. Official release notes date31March; evaluator release dates may say2April and do not override the vendor date. Hosted endpoint precision/provider conditions are separate from reference-weight benchmark evidence.

Compare Gemma 4 31B with other models →Compare subscription plans Explore data coverage

Pricing · OpenRouter

Dated cached OpenRouter rates in USD per 1M tokens. Open the dashboard for live enhancements. Per-metric endpoint minima can refer to different providers; they are not a guaranteed combined rate from one endpoint.

Recorded pricing tiers
Tier	Input / 1M	Output / 1M	Cached input / 1M	Cache write / 1M	Date & source
Default	$0.1	$0.34	$0.012	—	2026-10-20 · OpenRouter source

Recorded pricing notes

Min-healthy endpoint minima checked19October2026: input $0.1/1M from CoreWeave (coreweave/fp4, 100% 30m uptime); output $0.32/1M from DeepInfra (deepinfra/fp8/flex, 100% 30m uptime); cached $0.012/1M from Chutes (chutes/fp4, 100% 30m uptime). Each metric may name a different provider; these are not one combined endpoint quote. Published discounted rates are already inclusive; flags are not applied a second time. Hosted precision/service tiers do not establish reference BF16 benchmark precision. No contributor sibling found in all53-model audit. Keyless API throughput is unavailable, not zero.; min-healthy endpoint minima 2026-10-19; min-healthy endpoint minima 2026-10-20

Official / vendor benchmarks

Default headline records. Own-vendor, peer-vendor and third-party provenance remain visible in evidence; configurations may differ.

Official / vendor headline scores; expand evidence for every record
Benchmark / evaluator	Headline score	Evidence
MMLU-Pro	85.2%unknown effort	All 1 recorded result & sources 85.2% · raw 85.2 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
AIME 2026	89.2%unknown effort	All 1 recorded result & sources 89.2% · raw 89.2 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. No tools. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
LiveCodeBench v6	80%unknown effort	All 1 recorded result & sources 80% · raw 80 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
Codeforces Rating	2150unknown effort	All 1 recorded result & sources 2150 · raw 2150 rating Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
SciCode (Gemma report; aggregation unspecified)	43%unknown effort	All 1 recorded result & sources 43% · raw 43 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
GPQA Diamond	84.3%unknown effort	All 1 recorded result & sources 84.3% · raw 84.3 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
BigBench Extra Hard (micro average)	74.4%unknown effort	All 1 recorded result & sources 74.4% · raw 74.4 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
Humanity's Last Exam (no tools)	19.5%unknown effort	All 1 recorded result & sources 19.5% · raw 19.5 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
Humanity's Last Exam (with tools)	26.5%unknown effort	All 1 recorded result & sources 26.5% · raw 26.5 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Search tool enabled. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
IFBench (Gemma report protocol)	76%unknown effort	All 1 recorded result & sources 76% · raw 76 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. IFBench strict/loose convention is not stated in this table. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
IFEval	98.9%unknown effort	All 1 recorded result & sources 98.9% · raw 98.9 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
MMMLU	88.4%unknown effort	All 1 recorded result & sources 88.4% · raw 88.4 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
MRCR v2 (8 needles,128K average)	66.4%unknown effort	All 1 recorded result & sources 66.4% · raw 66.4 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
Terminal Bench Hard (Gemma report)	36%unknown effort	All 1 recorded result & sources 36% · raw 36 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
τ²-bench (Airline)	75%unknown effort	All 1 recorded result & sources 75% · raw 75 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
τ²-bench (Retail)	86.4%unknown effort	All 1 recorded result & sources 86.4% · raw 86.4 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
τ²-bench (Telecom)	69.3%unknown effort	All 1 recorded result & sources 69.3% · raw 69.3 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Report dated19June2026, not execution date. Visually checked numeric table. https://arxiv.org/pdf/2607.02770#page=5
MMMU Pro (no tools)	76.9%unknown effort	All 1 recorded result & sources 76.9% · raw 76.9 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native %. https://arxiv.org/pdf/2607.02770#page=6
MMMU Pro (Gemma,280 visual tokens)	75.8%unknown effort	All 1 recorded result & sources 75.8% · raw 75.8 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table12; thinking,280 visual tokens; separate visual-resolution configuration, not a new checkpoint. https://arxiv.org/pdf/2607.02770#page=17
MATH-Vision	85.6%unknown effort	All 1 recorded result & sources 85.6% · raw 85.6 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native %. https://arxiv.org/pdf/2607.02770#page=6
MATH-Vision (Gemma,280 visual tokens)	83.4%unknown effort	All 1 recorded result & sources 83.4% · raw 83.4 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table12; thinking,280 visual tokens; separate visual-resolution configuration, not a new checkpoint. https://arxiv.org/pdf/2607.02770#page=17
MedXpertQA MM	61.3%unknown effort	All 1 recorded result & sources 61.3% · raw 61.3 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native %. https://arxiv.org/pdf/2607.02770#page=6
MedXpertQA MM (Gemma,280 visual tokens)	60.7%unknown effort	All 1 recorded result & sources 60.7% · raw 60.7 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table12; thinking,280 visual tokens; separate visual-resolution configuration, not a new checkpoint. https://arxiv.org/pdf/2607.02770#page=17
InfographicVQA	92%unknown effort	All 1 recorded result & sources 92% · raw 92 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native %. https://arxiv.org/pdf/2607.02770#page=6
InfographicVQA (Gemma,280 visual tokens)	82.8%unknown effort	All 1 recorded result & sources 82.8% · raw 82.8 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table12; thinking,280 visual tokens; separate visual-resolution configuration, not a new checkpoint. https://arxiv.org/pdf/2607.02770#page=17
OmniDocBench1.5 (average edit distance)	0.131 edit distanceunknown effort	All 1 recorded result & sources 0.131 edit distance · raw 0.131 edit distance Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native edit distance. https://arxiv.org/pdf/2607.02770#page=6
OmniDocBench1.5 (average edit distance) (Gemma,280 visual tokens)	0.201 edit distanceunknown effort	All 1 recorded result & sources 0.201 edit distance · raw 0.201 edit distance Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table12; thinking,280 visual tokens; separate visual-resolution configuration, not a new checkpoint. https://arxiv.org/pdf/2607.02770#page=17
RULER32K (Gemma,no thinking)	96.8%none effort	All 1 recorded result & sources 96.8% · raw 96.8 % Headline · none effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table9, thinking disabled. RULER32K. Native %. https://arxiv.org/pdf/2607.02770#page=8
RULER128K (Gemma,no thinking)	96.4%none effort	All 1 recorded result & sources 96.4% · raw 96.4 % Headline · none effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table9, thinking disabled. RULER128K. Native %. https://arxiv.org/pdf/2607.02770#page=8
LOFT text retrieval128K (Recall@k) (Gemma,no thinking)	79.5%none effort	All 1 recorded result & sources 79.5% · raw 79.5 % Headline · none effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table9, thinking disabled. LOFT text retrieval128K (Recall@k). Native %. https://arxiv.org/pdf/2607.02770#page=8
GraphWalks <128K (F1) (Gemma,no thinking)	82.3%none effort	All 1 recorded result & sources 82.3% · raw 82.3 % Headline · none effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table9, thinking disabled. GraphWalks <128K (F1). Native %. https://arxiv.org/pdf/2607.02770#page=8
MTOB English→Kalamang ~128K half-book (Gemma,no thinking)	52.9 chrFnone effort	All 1 recorded result & sources 52.9 chrF · raw 52.9 chrF Headline · none effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table9, thinking disabled. MTOB English→Kalamang ~128K half-book. Native chrF. https://arxiv.org/pdf/2607.02770#page=8
MTOB English→Kalamang ~256K full-book (Gemma,no thinking)	54.3 chrFnone effort	All 1 recorded result & sources 54.3 chrF · raw 54.3 chrF Headline · none effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table9, thinking disabled. MTOB English→Kalamang ~256K full-book. Native chrF. https://arxiv.org/pdf/2607.02770#page=8
MTOB Kalamang→English ~128K half-book (Gemma,no thinking)	48.6 chrFnone effort	All 1 recorded result & sources 48.6 chrF · raw 48.6 chrF Headline · none effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table9, thinking disabled. MTOB Kalamang→English ~128K half-book. Native chrF. https://arxiv.org/pdf/2607.02770#page=8
MTOB Kalamang→English ~256K full-book (Gemma,no thinking)	46.2 chrFnone effort	All 1 recorded result & sources 46.2 chrF · raw 46.2 chrF Headline · none effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table9, thinking disabled. MTOB Kalamang→English ~256K full-book. Native chrF. https://arxiv.org/pdf/2607.02770#page=8
τ²-bench (Tau-2)	76.9%unknown effort	All 1 recorded result & sources 76.9% · raw 76.9 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google official card; published average over3 domains, thinking on. Not recomputed from report domains. https://huggingface.co/google/gemma-4-31B-it/blob/842da3794eaa0b77d5f08bae87a17459d91ff475/README.md
Arena Text (Google-cited19June2026)	1451unknown effort	All 1 recorded result & sources 1451 · raw 1451 Elo Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google report Table4; human Arena ratings as of19June2026,95%CI±8; model execution dates/quantization unpublished. https://arxiv.org/pdf/2607.02770#page=4
Arena Text (Google-cited2April2026)	1452unknown effort	All 1 recorded result & sources 1452 · raw 1452 Elo Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: Google launch figure; source says as of4/2/26. Visually checked exact printed value; no estimate from unlabeled plot. https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/
MMMLU (Gemma launch figure label)	85.2%unknown effort	All 1 recorded result & sources 85.2% · raw 85.2 % Headline · unknown effort · Own vendor Source/record date: 2026-10-19 Harness: literal Google launch figure label; benchmark-name conflict with full card/report; this cell is not used as ordinary MMLU-Pro or MMMLU. https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/

Independent evaluators

Evaluator harnesses are distinct from vendor measurements. Missing coverage is not a failed test.

Independent evaluator headline scores; expand evidence for every record
Benchmark / evaluator	Headline score	Evidence
GDPval-AA v2.1 Elo · Artificial Analysis	620unknown effort	All 2 recorded results & sources 620 · raw 620 Elo Headline · unknown effort · Independent evaluator Source/record date: 2026-10-20 Manually observed public GDPval-AA v2.1 leaderboard table, full 298-configuration pool checked 20 October 2026. Exact source configuration: Gemma 4 31B (Reasoning); creator: Google; source rank 211; source release-month label Apr 2026 is model release metadata, not execution date. Published rounded native Elo 620; literal displayed CI -20 / +20. Native Elo is not a percentage and is not the normalized (Elo−500)/2000 chart quantity. This versioned metric is separate from older generic GDPval-AA Elo and vendor GDPval/GDPval-AA results; no old evidence is overwritten. Execution date, inference precision, unprinted effort budgets and task costs are unpublished in this captured table. Elo is relative to the dated comparison pool; drift from an earlier observation does not prove a new evaluation run. Source says Reasoning (on), not an API effort level; reasoning effort remains unknown. Headline: highest published native Elo for this exact approved model; tied values prefer existing featured effort then source order. https://artificialanalysis.ai/evaluations/gdpval-aa 547 · raw 547 Elo Alternative · none effort · Independent evaluator Source/record date: 2026-10-20 Manually observed public GDPval-AA v2.1 leaderboard table, full 298-configuration pool checked 20 October 2026. Exact source configuration: Gemma 4 31B (Non-reasoning); creator: Google; source rank 218; source release-month label Apr 2026 is model release metadata, not execution date. Published rounded native Elo 547; literal displayed CI -23 / +23. Native Elo is not a percentage and is not the normalized (Elo−500)/2000 chart quantity. This versioned metric is separate from older generic GDPval-AA Elo and vendor GDPval/GDPval-AA results; no old evidence is overwritten. Execution date, inference precision, unprinted effort budgets and task costs are unpublished in this captured table. Elo is relative to the dated comparison pool; drift from an earlier observation does not prove a new evaluation run. https://artificialanalysis.ai/evaluations/gdpval-aa
Intelligence Index · Artificial Analysis	14.7unknown effort	All 1 recorded result & sources 14.7 · raw 14.6689500745828 index Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA-Omniscience Index · Artificial Analysis	-47.9unknown effort	All 2 recorded results & sources -47.9 · raw -47.93333333333333 index Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b -51.7 · raw -51.68333333333333 index Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
AA-Omniscience accuracy · Artificial Analysis	20%unknown effort	All 2 recorded results & sources 20% · raw 20.033333333333335 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 16.6% · raw 16.633333333333333 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
AA-Omniscience hallucination rate · Artificial Analysis	85%unknown effort	All 2 recorded results & sources 85% · raw 84.99374739474781 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 81.9% · raw 81.94722111155538 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
GDPval-AA Elo · Artificial Analysis	619.8unknown effort	All 2 recorded results & sources 619.8 · raw 619.79 Elo Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 547.4 · raw 547.44 Elo Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
Terminal-Bench 4.0 (AA) · Artificial Analysis	0%unknown effort	All 1 recorded result & sources 0% · raw 0 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
Terminal-Bench 2.1 (AA) · Artificial Analysis	43.4%unknown effort	All 2 recorded results & sources 43.4% · raw 43.4456928838951 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 29.2% · raw 29.213483146067396 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
Terminal Bench Hard (AA) · Artificial Analysis	36.4%unknown effort	All 2 recorded results & sources 36.4% · raw 36.363636363636395 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 30.3% · raw 30.303030303030297 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
SciCode (AA) · Artificial Analysis	45.5%unknown effort	All 1 recorded result & sources 45.5% · raw 45.4861111111111 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA-LCR · Artificial Analysis	69.7%unknown effort	All 2 recorded results & sources 69.7% · raw 69.6666666666667 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 46.7% · raw 46.6666666666667 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
IFBench (AA) · Artificial Analysis	75.6%unknown effort	All 2 recorded results & sources 75.6% · raw 75.57823129251699 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 53.5% · raw 53.469387755102005 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
Humanity's Last Exam (AA harness) · Artificial Analysis	23.6%unknown effort	All 2 recorded results & sources 23.6% · raw 23.6329935125116 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 11.8% · raw 11.8164967562558 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
GPQA Diamond (AA) · Artificial Analysis	85.7%unknown effort	All 2 recorded results & sources 85.7% · raw 85.6565656565657 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 76.3% · raw 76.2626262626263 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
CritPt (AA) · Artificial Analysis	1.4%unknown effort	All 2 recorded results & sources 1.4% · raw 1.42857142857143 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 0% · raw 0 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
GDP.PDF (AA) · Artificial Analysis	6%unknown effort	All 1 recorded result & sources 6% · raw 6 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AutomationBench (AA) · Artificial Analysis	4.9%unknown effort	All 1 recorded result & sources 4.9% · raw 4.933063448681232 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
MMMU-Pro (AA) · Artificial Analysis	73.4%unknown effort	All 2 recorded results & sources 73.4% · raw 73.4104046242775 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 70.3% · raw 70.3468208092486 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
τ²-Telecom (AA) · Artificial Analysis	59.9%unknown effort	All 2 recorded results & sources 59.9% · raw 59.9415204678363 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 65.5% · raw 65.4970760233918 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
τ³-Banking (AA) · Artificial Analysis	14.8%unknown effort	All 2 recorded results & sources 14.8% · raw 14.8453608247423 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 8.9% · raw 8.8659793814433 % Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
ITBench-AA · Artificial Analysis	37.3%unknown effort	All 1 recorded result & sources 37.3% · raw 37.288135593220304 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
EnterpriseOps-Gym-AA · Artificial Analysis	28.3%unknown effort	All 1 recorded result & sources 28.3% · raw 28.349746344374815 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA Industry Index · Finance and accounting · Artificial Analysis	13.6unknown effort	All 1 recorded result & sources 13.6 · raw 13.5684761197161 index Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA Industry Index · Strategy and operations · Artificial Analysis	9.3unknown effort	All 1 recorded result & sources 9.3 · raw 9.32330845317596 index Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA Industry Index · Legal · Artificial Analysis	13.9unknown effort	All 1 recorded result & sources 13.9 · raw 13.9375413276838 index Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA Industry Index · Engineering · Artificial Analysis	16.7unknown effort	All 1 recorded result & sources 16.7 · raw 16.7497847411625 index Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA Industry Index · Economics · Artificial Analysis	20.6unknown effort	All 1 recorded result & sources 20.6 · raw 20.5905685627124 index Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA-Briefcase Elo · Artificial Analysis	367.9unknown effort	All 1 recorded result & sources 367.9 · raw 367.87 Elo Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA-Briefcase analytical quality Elo · Artificial Analysis	128.2unknown effort	All 1 recorded result & sources 128.2 · raw 128.2 Elo Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA-Briefcase presentation Elo · Artificial Analysis	330.2unknown effort	All 1 recorded result & sources 330.2 · raw 330.23 Elo Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
AA-Briefcase rubric pass rate · Artificial Analysis	7.8%unknown effort	All 1 recorded result & sources 7.8% · raw 7.777777777777778 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b
Output speed · Artificial Analysis	35.7 tok/sunknown effort	All 2 recorded results & sources 35.7 tok/s · raw 35.6741466279495 tok/s Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Reasoning enabled without a published common API effort label. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b 48.5 tok/s · raw 48.5325324993434 tok/s Alternative · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
Intelligence Index (estimated) · Artificial Analysis	13.9none effort	All 1 recorded result & sources 13.9 · raw 13.9198819005287 index Headline · none effort · Independent evaluator Source/record date: 2026-10-19 Independent AA exact model page explicitly links https://huggingface.co/google/gemma-4-31B-it. Explicit non-reasoning configuration. Index v4.3.2; inference precision, execution date and exact budget unpublished. Linking reference weights does not prove BF16 inference. Costs with unavailable/zero model prices are not imported as free tariffs. https://artificialanalysis.ai/models/gemma-4-31b-non-reasoning
MortgageTax · Vals AI	61.4%high effort	All 1 recorded result & sources 61.4% · raw 61.37 % Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Independent Vals current model-page card; MortgageTax. Published uncertainty±0.97; rank60/98 at source check. Default Google,temperature1,max output32768,reasoning high as published. Individual benchmarks may override defaults. https://www.vals.ai/models/google_gemma-4-31b-it
SAGE · Vals AI	55%high effort	All 1 recorded result & sources 55% · raw 55.03 % Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Independent Vals current model-page card; SAGE. Published uncertainty±3.41; rank2/91 at source check. Default Google,temperature1,max output32768,reasoning high as published. Individual benchmarks may override defaults. https://www.vals.ai/models/google_gemma-4-31b-it
Terminal-Bench 2.0 · Vals AI	39.3%high effort	All 1 recorded result & sources 39.3% · raw 39.33 % Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Independent Vals current model-page card; Terminal-Bench 2.0. Published uncertainty±5.21; rank35/67 at source check. Default Google,temperature1,max output32768,reasoning high as published. Individual benchmarks may override defaults. https://www.vals.ai/models/google_gemma-4-31b-it
Vals Index (9 April 2026 snapshot) · Vals AI	38.9%unknown effort	All 1 recorded result & sources 38.9% · raw 38.94 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Dated Vals9April2026 update on the exact Gemma4 31B IT page. Historical suite/population; do not merge with current Vals Index or full SWE-bench. Execution date and historical effort not restated. https://www.vals.ai/models/google_gemma-4-31b-it
Vals Multimodal Index (9 April 2026 snapshot) · Vals AI	45.1%unknown effort	All 1 recorded result & sources 45.1% · raw 45.12 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Dated Vals9April2026 update on the exact Gemma4 31B IT page. Historical suite/population; do not merge with current Vals Index or full SWE-bench. Execution date and historical effort not restated. https://www.vals.ai/models/google_gemma-4-31b-it
Case Law v2 (9 April 2026 snapshot) · Vals AI	52.6%unknown effort	All 1 recorded result & sources 52.6% · raw 52.63 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Dated Vals9April2026 update on the exact Gemma4 31B IT page. Historical suite/population; do not merge with current Vals Index or full SWE-bench. Execution date and historical effort not restated. https://www.vals.ai/models/google_gemma-4-31b-it
Finance Agent v1.1 (9 April 2026 snapshot) · Vals AI	50.8%unknown effort	All 1 recorded result & sources 50.8% · raw 50.79 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Dated Vals9April2026 update on the exact Gemma4 31B IT page. Historical suite/population; do not merge with current Vals Index or full SWE-bench. Execution date and historical effort not restated. https://www.vals.ai/models/google_gemma-4-31b-it
Corp Fin v2 shared-max-context (9 April 2026 snapshot) · Vals AI	59.7%unknown effort	All 1 recorded result & sources 59.7% · raw 59.67 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Dated Vals9April2026 update on the exact Gemma4 31B IT page. Historical suite/population; do not merge with current Vals Index or full SWE-bench. Execution date and historical effort not restated. https://www.vals.ai/models/google_gemma-4-31b-it
SWE-bench Verified subset (9 April 2026 snapshot) · Vals AI	53.9%unknown effort	All 1 recorded result & sources 53.9% · raw 53.92 % Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Dated Vals9April2026 update on the exact Gemma4 31B IT page. Historical suite/population; do not merge with current Vals Index or full SWE-bench. Execution date and historical effort not restated. https://www.vals.ai/models/google_gemma-4-31b-it
Bug Hunt Bench · Bugs fixed (105 total) · Bug Hunt Bench	4 fixesunknown effort	All 2 recorded results & sources 4 fixes · raw 4 fixes Headline · unknown effort · Independent evaluator Source/record date: 2026-10-19 Harness: Claude Code / OpenRouter. ONE RUN PER REPO, not a replicated mean - read the 4/105 as a single data point for this model, not a settled estimate. The repo 1 and repo 2 legs ran under DIFFERENT CONTEXT REGIMES (repo 1 compacted three times, repo 2 zero), so the combined total mixes two conditions rather than two comparable draws of the same one; the per-repo split is the more honest read. NOT FEATURED: mean rows and n=1 rows are not featured by default on this board unless opted in, and this one was not. First appearance of Gemma 4 31B on this board - native bf16, reached through OpenRouter pinned to one host, Crusoe, fallbacks off (Antigravity does not serve Gemma at all - checked live against 15 models offered, none of them Gemma). 4 of 105 (repo 1: 1, repo 2: 3), 2 genuine unplanted extras (one per repo), and ZERO claimed-only fixes on either repo - nothing it reported went unconfirmed by the blind judge. * THE TWO LEGS DID NOT RUN IN THE SAME REGIME, AND THIS TIME THE CAUSE IS VERIFIED, NOT JUST FLAGGED * the repo 1 leg compacted three times, peaking at 198,297 tokens (triggers at 198297, 196796 and 183827) - the repo 2 leg shows zero compactions. That fully accounts for the wall and cost gap between them: repo 1 ran 205.2 minutes for 8.0811 dollars against repo 2's 16.6 minutes for 0.5261 - roughly 12x the wall and 15x the cost for one more repo's worth of the same model on the same day. Repeated auto-compaction means the model spent much of that run re-reading context it had already seen, which is the direct mechanical explanation for the token and cost gap, not model instability or provider throttling. * MACHINE CONTENTION ALSO TOUCHED THE REPO 1 WALL FIGURE, SEPARATELY * that leg's clean attempt ran while five other legs shared the box (queued replication work for other arms); contention can inflate wall further but, per this board's standing note, does not touch scores, tokens or cost - the compaction count is the load-bearing explanation for those. A first, EARLIER attempt at this same repo 1 leg ran into the same Luna-adjacent contention window and exited non-zero; it is not this row and cost 0.1943 dollars of its own, separately absorbed, not counted here. BILLING: real OpenRouter bill (credits-delta) via meter=openrouter on both legs, not a list-rate estimate. EFFORT NOT ASSERTED: OpenRouter does not implement a tier field this route honors, so none was requested and none is claimed. Native planted fixes, not percentages. Quantized/provider configurations remain distinct; default is source-featured Crusoe observation. https://github.com/phuryn/bug-hunt-bench/blob/ff714353baae81f0d8bcff80fdf96c1468662d44/data/benchmark.json 3 fixes · raw 3 fixes Alternative · unknown effort · Independent evaluator Source/record date: 2026-10-19 Harness: Claude Code / OpenRouter. ONE RUN PER REPO, not a replicated mean - read the 3/105 as a single data point, not a settled estimate. The dramatic cost gap against the native sibling is READ FROM ONE RUN EACH, not a controlled quantization sweep - fp4 happened not to compact here while bf16 did on repo 1, but nothing here proves fp4 never compacts or bf16 always does. NOT FEATURED: mean rows and n=1 rows are not featured by default on this board unless opted in, and this one was not. The 4-bit fp4 sibling of the native row above, reached through OpenRouter pinned to CoreWeave (the cheapest tools-capable fp4 host for this slug - DeepInfra fp4 is cheaper still but reports tools=False, disqualified, same shape as the Parasail-vs-DeepInfra choice on the Qwen3.8-27B 8-bit row). 3 of 105 (repo 1: 1, repo 2: 2), 2 genuine unplanted extras (both on repo 2), ZERO claimed-only fixes on either repo. * ESSENTIALLY THE SAME SCORE AS NATIVE, AT A FRACTION OF THE COST * 3/105 here against native's 4/105, but 0.56 dollars combined against native's 8.61 - roughly 1/15th the cost for one point less, because NEITHER leg here hit the compaction native hit on repo 1 (zero compactions on both this arm's legs). Both legs ran clean and quick: 12.8 and 12.1 minutes, 0.2581 and 0.3034 dollars. BILLING: real OpenRouter bill (credits-delta) via meter=openrouter on both legs, not a list-rate estimate. EFFORT NOT ASSERTED, same reason as the native row: OpenRouter does not implement a tier field this route honors. Native planted fixes, not percentages. Quantized/provider configurations remain distinct; default is source-featured Crusoe observation. https://github.com/phuryn/bug-hunt-bench/blob/ff714353baae81f0d8bcff80fdf96c1468662d44/data/benchmark.json

Read how we select and source scores or the comparison guide.