Examenos

59 · raw 59 index

Headline · high effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Sourced by Google from the Artificial Analysis Intelligence Index v5.0 (with tools); AA ran Carbon at high reasoning. Independent AA observation is in the evaluator data.

70.4% · raw 70.4 %

Headline · high effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Vendor-reported Vals Index v2.2 figure; Vals ran Carbon at high reasoning. Vals primary run lives in the evaluator data.

9.6% · raw 9.6 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Self-computed with a mini-swe-agent harness, subtask-weighted reward. Gemini API with the highest thinking settings per Google's methodology.

61.2% · raw 61.2 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Proximal official public leaderboard. Gemini API with the highest thinking settings per Google's methodology.

65.2% · raw 65.2 %

Headline · high effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Sourced by Google from Artificial Analysis Terminal-Bench 4.0; AA ran Carbon at high reasoning.

77.4% · raw 77.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Self-computed on 122 v1.1 tasks, ALE-Claw harness, 5-hour window, 17 October task version without the 26 October update, safety filters on (flagged responses returned empty); averaged rubric score. Gemini API with the highest thinking settings per Google's methodology.

81.5% · raw 81.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Self-computed on the 82-task offline subset, partial score, maximum over 3 runs (single attempt each); updated Gemini CUA harness. Gemini API with the highest thinking settings per Google's methodology.

53.9% · raw 53.9 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Private set via the official Zapier public leaderboard. Gemini API with the highest thinking settings per Google's methodology.

72.5% · raw 72.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Without tools, following the official Surge public leaderboard. Gemini API with the highest thinking settings per Google's methodology.

96.2 f1 · raw 96.2 f1

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Self-computed on the 200 problems between 256k and 1M tokens, BFS F1. Gemini API with the highest thinking settings per Google's methodology.

80% · raw 80 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Official Surge public leaderboard. Gemini API with the highest thinking settings per Google's methodology.

64.8% · raw 64.8 %

Headline · high effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Sourced by Google from Artificial Analysis Terminal-Bench Science 0.1; AA ran Carbon at high reasoning.

54.8% · raw 54.8 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Harness: Antigravity, 10 h extended timeout per task. Self-computed with the open-source setup, single attempt, no internet, mitigations off; solved only when the exploit captures the flag and an LLM judge verifies the intended vulnerability. Main-table value. Gemini API with the highest thinking settings per Google's methodology.

53.5% · raw 53.5 %

Alternative · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Harness: Antigravity, default 6 h timeout per task. Same setup as the 10 h headline. Gemini API with the highest thinking settings per Google's methodology.

88.2% · raw 88.2 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Self-computed pass@1 on the 183-task May 2026 JavaScript engine split, averaged over 3 attempts per task, Antigravity harness; solved if the PoC crashes the vulnerable build but not the patched build, or a judge confirms the intended vulnerability. Gemini API with the highest thinking settings per Google's methodology.

0% · raw 0 %

Headline · max effort · Own vendor

Source/record date: 2026-10-28

Gemini 4 Carbon launch table (Google, 28 October 2026); setup from Google's evaluation methodology PDF https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_model_evaluation.pdf. Gray Swan indirect prompt injection attack success rate at k=15 (lower is better); chart label 0%. Gemini API with the highest thinking settings per Google's methodology.

71% · raw 71 %

Headline · high effort · Own vendor

Source/record date: 2026-10-28

Harness: Antigravity. Collinear CWE-bench v1 board (updated 28 October 2026), board label "Gemini 4 Carbon", harness Antigravity; 120 held-out tasks, four one-hour rollouts per task at high reasoning.

81% · raw 81 %

Headline · high effort · Own vendor

Source/record date: 2026-10-28

Harness: Antigravity. Collinear CWE-bench v1 board (updated 28 October 2026), board label "Gemini 4 Carbon", harness Antigravity; 120 held-out tasks, four one-hour rollouts per task at high reasoning.

71% · raw 71 %

Headline · high effort · Own vendor

Source/record date: 2026-10-28

Harness: Antigravity. Collinear CWE-bench v1 board (updated 28 October 2026), board label "Gemini 4 Carbon", harness Antigravity; 120 held-out tasks, four one-hour rollouts per task at high reasoning.

83% · raw 83 %

Headline · high effort · Own vendor

Source/record date: 2026-10-28

Harness: Antigravity. Collinear CWE-bench v1 board (updated 28 October 2026), board label "Gemini 4 Carbon", harness Antigravity; 120 held-out tasks, four one-hour rollouts per task at high reasoning.

76.468% · raw 76.468 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$5.755811/task · raw 5.755811 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1648.590 s · raw 1648.59 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

81.582% · raw 81.582 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

71.068% · raw 71.068 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

65.564% · raw 65.564 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

68.419% · raw 68.419 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

86.302% · raw 86.302 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

70.534% · raw 70.534 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

83.067% · raw 83.067 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

87.856% · raw 87.856 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

73.536% · raw 73.536 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

66.350% · raw 66.35 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$6.462098/task · raw 6.462098 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1838.140 s · raw 1838.14 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

67.162% · raw 67.162 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

76.468% · raw 76.468 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

48.860% · raw 48.86 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

67.162% · raw 67.162 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$7.290456/task · raw 7.290456 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1779.586 s · raw 1779.586 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

55.945% · raw 55.945 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

76.240% · raw 76.24 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

81.637% · raw 81.637 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

65.972% · raw 65.972 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

62.704% · raw 62.704 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

34.750% · raw 34.75 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

69.833% · raw 69.833 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

82.360% · raw 82.36 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

80.972% · raw 80.972 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

49.991% · raw 49.991 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

17.500% · raw 17.5 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$6.515809/task · raw 6.515809 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1720.841 s · raw 1720.841 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

20.000% · raw 20 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

50.000% · raw 50 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

20.000% · raw 20 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

30.000% · raw 30 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

40.000% · raw 40 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

40.000% · raw 40 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

20.000% · raw 20 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

30.000% · raw 30 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

30.000% · raw 30 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

40.000% · raw 40 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

30.000% · raw 30 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

94.777% · raw 94.777 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

96.739% · raw 96.739 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

93.548% · raw 93.548 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

97.015% · raw 97.015 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

92.734% · raw 92.734 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

96.275% · raw 96.275 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

95.034% · raw 95.034 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

96.463% · raw 96.463 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

97.703% · raw 97.703 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

95.563% · raw 95.563 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

88.390% · raw 88.39 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

97.360% · raw 97.36 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

91.577% · raw 91.577 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

91.108% · raw 91.108 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

95.805% · raw 95.805 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

95.455% · raw 95.455 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

94.669% · raw 94.669 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

96.619% · raw 96.619 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

93.227% · raw 93.227 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

97.180% · raw 97.18 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

97.668% · raw 97.668 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

91.935% · raw 91.935 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

93.645% · raw 93.645 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

94.084% · raw 94.084 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

94.781% · raw 94.781 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

70.355% · raw 70.355 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$10.508065/task · raw 10.508065 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2864.864 s · raw 2864.864 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

89.060% · raw 89.06 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: OpenHands. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

67.162% · raw 67.162 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

76.468% · raw 76.468 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

80.051% · raw 80.051 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Index Code Migration is the 60-task subset (50 CLI + 10 COBOL), not standalone Migration.

55.769% · raw 55.769 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

17.500% · raw 17.5 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

63.131% · raw 63.131 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

48.860% · raw 48.86 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

75.828% · raw 75.828 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$7.199786/task · raw 7.199786 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2306.554 s · raw 2306.554 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

70.776% · raw 70.776 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

71.756% · raw 71.756 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

78.377% · raw 78.377 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

97.668% · raw 97.668 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

88.390% · raw 88.39 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

94.084% · raw 94.084 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

45.455% · raw 45.455 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

77.215% · raw 77.215 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

71.825% · raw 71.825 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

55.769% · raw 55.769 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

94.777% · raw 94.777 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

55.769% · raw 55.769 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$8.743524/task · raw 8.743524 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

3112.579 s · raw 3112.579 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

65.152% · raw 65.152 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

46.512% · raw 46.512 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

48.148% · raw 48.148 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

55.263% · raw 55.263 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

59.649% · raw 59.649 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

45.455% · raw 45.455 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

70.000% · raw 70 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

45.455% · raw 45.455 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

91.609% · raw 91.609 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

92.520% · raw 92.52 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

90.979% · raw 90.979 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

96.504% · raw 96.504 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

91.233% · raw 91.233 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

90.590% · raw 90.59 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

84.761% · raw 84.761 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

96.764% · raw 96.764 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

87.445% · raw 87.445 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

48.860% · raw 48.86 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$6.340026/task · raw 6.340026 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2086.243 s · raw 2086.243 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

42.005% · raw 42.005 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

68.810% · raw 68.81 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

53.861% · raw 53.861 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

65.428% · raw 65.428 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

40.461% · raw 40.461 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

25.566% · raw 25.566 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

71.825% · raw 71.825 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

70.357% · raw 70.357 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

73.985% · raw 73.985 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

71.812% · raw 71.812 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

82.983% · raw 82.983 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

65.817% · raw 65.817 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

66.633% · raw 66.633 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

63.131% · raw 63.131 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$7.452059/task · raw 7.452059 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2706.770 s · raw 2706.77 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

62.963% · raw 62.963 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

59.524% · raw 59.524 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

78.788% · raw 78.788 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

44.444% · raw 44.444 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

60.000% · raw 60 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

73.333% · raw 73.333 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

66.667% · raw 66.667 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: published Vals component protocol. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

89.060% · raw 89.06 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: OpenHands. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates.

$8.254926/task · raw 8.254926 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: OpenHands. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2172.019 s · raw 2172.019 s

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: OpenHands. Google API, high reasoning effort, max_output_tokens 1048576; temperature and top-p not published (Google defaults). Limited-access Fairwind model. Preserve benchmark version and population, task versus criterion pass, and weighted versus all-pass credit. Standard errors are not confidence intervals. Serving checkpoint hash and run dates unpublished; source update/check dates are not inference dates. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

$15984.21 · raw 15984.21 $

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-29

Vending-Bench 2 net gain = displayed average Money Balance $16,484.21 (± $1,096) minus the $500 starting balance, the convention of every stored Money gain row. Rank 1 on the 29 October board; Andon does not label the reasoning setting.

58.6% · raw 58.6 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-29

Blueprint-Bench 2 connectivity similarity; published fractional score 0.586 multiplied by 100. Ties the board's human baseline.

59 · raw 59 index

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Rank 1 on v5.0 at the post date. Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

$7.76/task · raw 7.76 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Standard-pricing usage cost per v5.0 task; not a token tariff. Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

123 k tokens/task · raw 123 k tokens/task

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Displayed as "123k output tokens per task". Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

59 · raw 59 index

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Harness: Antigravity CLI. Model-plus-agent measurement; AA's same-day Copilot Fusion post identifies the current index as Coding Agent Index v2.0. Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

1609 · raw 1609 Elo

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

1598 · raw 1598 Elo

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

1671 · raw 1671 Elo

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

54 · raw 54 index

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Signed knowledge index; v1.1 (3,000 questions) is not comparable with v1.0 rows. Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

58% · raw 58 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

11% · raw 11 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Lower is better. Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

89% · raw 89 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Chart label; equals 100 minus the 11% hallucination rate in the post text. Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

78% · raw 78 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

65% · raw 65 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

41% · raw 41 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

AA harness; separate from Cognition's own FrontierCode 2.0 board. Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

69% · raw 69 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

New v5.0 component; not Humanity's Last Exam (AA). Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

28% · raw 28 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

All-pass. Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

65% · raw 65 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.

63% · raw 63 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-29

CritPt v2 hard subset; not the earlier CritPt (AA) rows. Manual observation of Artificial Analysis's public 28 October 2026 Gemini 4 Carbon post and its "Full benchmark results" chart image (visually inspected; research_notes/Model and benchmark news October 29/sources/x/). Configuration "Gemini 4 Carbon (high)". AA notes the model is not publicly available and that its figures use standard $4/$20 pricing despite Google's 50% launch discount. Intelligence Index v5.0 (27 October 2026) components; values are AA's displayed rounding. No API, internal app state, scheduled import, hidden precision or inferred run date.