Examenos

50.3% · raw 50.3 %

Headline · max effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI launch evaluation; exact agent version and execution date unpublished. Vendor result, not the independent evaluator run.

62.7% · raw 62.7 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI system-card HealthBench. Length-adjusted score; raw score 64.2%, mean response length 2531 characters. Effort and run date unpublished.

55.1% · raw 55.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI system-card HealthBench. Length-adjusted score; raw score 51.1%, mean response length 1331 characters. Effort and run date unpublished.

31.4% · raw 31.4 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI system-card HealthBench. Length-adjusted score; raw score 26.0%, mean response length 1306 characters. Effort and run date unpublished.

95.1% · raw 95.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI system-card HealthBench. Length-adjusted score; raw score 94.8%, mean response length 1312 characters. Effort and run date unpublished.

55.5% · raw 55.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI MentalHealthBench, max effort; published ±1.0 standard error. Four responses per task; overall population 1,215 tasks. Not clinical advice.

56.5% · raw 56.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI MentalHealthBench, max effort; published ±1.3 standard error. Four responses per task; overall population 1,215 tasks. Not clinical advice.

56.5% · raw 56.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI MentalHealthBench, max effort; published ±2.2 standard error. Four responses per task; overall population 1,215 tasks. Not clinical advice.

52.9% · raw 52.9 %

Headline · max effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI MentalHealthBench, max effort; published ±1.9 standard error. Four responses per task; overall population 1,215 tasks. Not clinical advice.

45.3% · raw 45.34 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI system-card KernelGen 1P; setup and run date as published, not KernelBench.

55.9% · raw 55.9 %

Alternative · low effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI Agents’ Last Exam v1 launch chart; partial-credit score, not pass rate. Chart-reported cost $0.16/task; model-plus-harness observation. OpenAI testing.

57.6% · raw 57.6 %

Alternative · medium effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI Agents’ Last Exam v1 launch chart; partial-credit score, not pass rate. Chart-reported cost $0.29/task; model-plus-harness observation. OpenAI testing.

57.2% · raw 57.2 %

Alternative · high effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI Agents’ Last Exam v1 launch chart; partial-credit score, not pass rate. Chart-reported cost $0.39/task; model-plus-harness observation. OpenAI testing.

58.5% · raw 58.5 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI Agents’ Last Exam v1 launch chart; partial-credit score, not pass rate. Chart-reported cost $0.66/task; model-plus-harness observation. OpenAI testing.

60% · raw 60 %

Headline · max effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI Agents’ Last Exam v1 launch chart; partial-credit score, not pass rate. Chart-reported cost $1.01/task; model-plus-harness observation. OpenAI testing.

97.1% · raw 97.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; ACE-normalized mean capability. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

83.3% · raw 83.33 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; pass@1 on the published JavaScript-engine assessment. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

24.5% · raw 24.5 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; OpenAI internal port arbitrary-code-execution success on the June–August 2026 cohort. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

76.4% · raw 76.39 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; mean rubric reward on the OpenAI internal research debugging evaluation. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

60% · raw 60 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; runtime category, 3/5 targets. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

14.3% · raw 14.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; parser category, 2/14 targets. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

0% · raw 0 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; egress proxy category, 0/3 targets. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

1.8% · raw 1.83 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; failure rate in broken search evaluation; lower is better. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

98.8% · raw 98.77 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; indirect prompt-injection robustness; exact rendered figure label. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

0% · raw 0 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-27

Harness: OpenAI GPT-6.1 Terra system-card evaluation; attempted or successful auto-review circumvention on the published setup; lower is better. Settings and exact execution date are not independently established. This is a vendor-reported result, not a general safety guarantee.

40 correct · raw 40 correct

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-27

Harness: Plotly Studio Embedded, one run on 43 private English data-analytics questions in a synthetic wind-energy warehouse. Claude Sonnet 5 judges against Plotly answer key. One or two questions can be noise; no error bars. Exact checkpoint hash, run date and effort unpublished. Company-operated analytics benchmark, distinct from provider results.

73.330% · raw 73.33 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

76.337% · raw 76.337 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

70.338% · raw 70.338 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.100% · raw 64.1 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.287% · raw 64.287 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

80.685% · raw 80.685 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

74.827% · raw 74.827 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

75.593% · raw 75.593 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.331% · raw 86.331 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.484% · raw 67.484 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

52.864% · raw 52.864 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

55.958% · raw 55.958 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.330% · raw 73.33 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

15.166% · raw 15.166 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

55.958% · raw 55.958 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

42.561% · raw 42.561 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.786% · raw 63.786 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

74.393% · raw 74.393 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

56.286% · raw 56.286 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.447% · raw 50.447 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

25.825% · raw 25.825 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.602% · raw 50.602 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.029% · raw 83.029 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.150% · raw 64.15 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

35.103% · raw 35.103 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

1.667% · raw 1.667 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

87.232% · raw 87.232 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.971% · raw 82.971 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.351% · raw 88.351 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.112% · raw 90.112 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.391% · raw 83.391 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.980% · raw 90.98 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

85.103% · raw 85.103 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.926% · raw 92.926 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

89.929% · raw 89.929 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.737% · raw 88.737 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.397% · raw 82.397 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.884% · raw 94.884 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

85.305% · raw 85.305 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.673% · raw 83.673 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.409% · raw 86.409 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

85.818% · raw 85.818 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.787% · raw 88.787 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.815% · raw 91.815 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

87.649% · raw 87.649 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.835% · raw 83.835 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.732% · raw 93.732 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.226% · raw 83.226 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

79.431% · raw 79.431 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.550% · raw 88.55 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

85.017% · raw 85.017 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

60.168% · raw 60.168 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.997% · raw 83.997 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

55.958% · raw 55.958 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.330% · raw 73.33 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

69.685% · raw 69.685 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

25.962% · raw 25.962 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

1.667% · raw 1.667 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

49.495% · raw 49.495 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

15.166% · raw 15.166 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

58.638% · raw 58.638 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

51.642% · raw 51.642 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.454% · raw 50.454 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.662% · raw 63.662 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.732% · raw 93.732 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.397% · raw 82.397 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.550% · raw 88.55 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

4.545% · raw 4.545 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

51.280% · raw 51.28 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

49.704% · raw 49.704 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

25.962% · raw 25.962 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

87.232% · raw 87.232 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

25.962% · raw 25.962 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

45.455% · raw 45.455 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

16.279% · raw 16.279 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

14.815% · raw 14.815 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

23.684% · raw 23.684 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

17.544% · raw 17.544 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

4.545% · raw 4.545 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

56.667% · raw 56.667 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

9.091% · raw 9.091 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

78.708% · raw 78.708 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.800% · raw 84.8 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

75.896% · raw 75.896 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

80.550% · raw 80.55 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

77.981% · raw 77.981 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

76.229% · raw 76.229 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.806% · raw 64.806 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.819% · raw 92.819 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

69.793% · raw 69.793 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

15.166% · raw 15.166 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

16.094% · raw 16.094 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

14.889% · raw 14.889 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

22.079% · raw 22.079 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

19.080% · raw 19.08 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

12.600% · raw 12.6 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

5.855% · raw 5.855 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

49.704% · raw 49.704 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

47.212% · raw 47.212 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.878% · raw 40.878 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

53.050% · raw 53.05 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.948% · raw 63.948 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

51.716% · raw 51.716 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

42.501% · raw 42.501 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

49.495% · raw 49.495 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.000% · raw 50 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

54.762% · raw 54.762 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

42.424% · raw 42.424 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.741% · raw 40.741 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

26.667% · raw 26.667 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

80.000% · raw 80 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

58.333% · raw 58.333 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.997% · raw 83.997 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.7 fixes · raw 40.7 fixes

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Harness: Codex CLI 0.159.0; mean of three runs with independently blind-graded reports. Judge grok-4.7 for this OpenAI model. Source values are bugs fixed out of 105, not percentages. Effort, wall time and list-price estimates describe this specific harness.

11.3 fixes · raw 11.3 fixes

Alternative · xhigh effort · Independent evaluator

Source/record date: 2026-10-27

Harness: Codex CLI 0.159.0; mean of three runs with independently blind-graded reports. Judge grok-4.7 for this OpenAI model. Source values are bugs fixed out of 105, not percentages. Effort, wall time and list-price estimates describe this specific harness.

9.7 fixes · raw 9.7 fixes

Alternative · high effort · Independent evaluator

Source/record date: 2026-10-27

Harness: Codex CLI 0.159.0; mean of three runs with independently blind-graded reports. Judge grok-4.7 for this OpenAI model. Source values are bugs fixed out of 105, not percentages. Effort, wall time and list-price estimates describe this specific harness.

48 · raw 48 index

Headline · max effort · Independent evaluator

Source/record date: 2026-10-27

Manual public rendered model-page value, checked 27 October. Artificial Analysis Intelligence Index v4.3.2, not task accuracy; serving checkpoint and exact run date unpublished.

45 · raw 45 index

Alternative · xhigh effort · Independent evaluator

Source/record date: 2026-10-27

Manual public rendered Artificial Analysis leaderboard/model-page label, 27 October 2026. Version 4.3.2 index, not task accuracy. Serving checkpoint and exact evaluation date unpublished. This is a separate effort configuration; max remains the default headline.

44 · raw 44 index

Alternative · high effort · Independent evaluator

Source/record date: 2026-10-27

Manual public rendered Artificial Analysis leaderboard/model-page label, 27 October 2026. Version 4.3.2 index, not task accuracy. Serving checkpoint and exact evaluation date unpublished. This is a separate effort configuration; max remains the default headline.

41 · raw 41 index

Alternative · medium effort · Independent evaluator

Source/record date: 2026-10-27

Manual public rendered Artificial Analysis leaderboard/model-page label, 27 October 2026. Version 4.3.2 index, not task accuracy. Serving checkpoint and exact evaluation date unpublished. This is a separate effort configuration; max remains the default headline.

36 · raw 36 index

Alternative · low effort · Independent evaluator

Source/record date: 2026-10-27

Manual public rendered Artificial Analysis leaderboard/model-page label, 27 October 2026. Version 4.3.2 index, not task accuracy. Serving checkpoint and exact evaluation date unpublished. This is a separate effort configuration; max remains the default headline.

33 · raw 33 index

Alternative · none effort · Independent evaluator

Source/record date: 2026-10-27

Manual public rendered Artificial Analysis leaderboard/model-page label, 27 October 2026. Version 4.3.2 index, not task accuracy. Serving checkpoint and exact evaluation date unpublished. This is a separate effort configuration; max remains the default headline.