Examenos

74.9% · raw 74.9 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “DeepSWE v1.1” in the Tencent Hy4 release appendix, column “Hy4”. mini-swe-agent (version unstated), 8 CPU cores and 16 GB per sandbox. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

43.8% · raw 43.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “SWE-Marathon v1.1*” in the Tencent Hy4 release appendix, column “Hy4”. Official settings except a doubled agent timeout; repeated until eight valid scores. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

30% · raw 30 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “ProgramBench (almost solved)*” in the Tencent Hy4 release appendix, column “Hy4”. Almost Solved score; Claude Code 2.1.220 for Hy4, peers on the scaffolds from their own blog posts or ProgramBench’s official mini-SWE-agent. Claude Opus 5.5 hit content-filter blocks on 41 instances. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

44.4% · raw 44.4 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Terminal-Bench 4.0” in the Tencent Hy4 release appendix, column “Hy4”. Claude Code harness, up to 500 turns and 13 hours per trial, 16 CPU / 32 GB, one H100 for the three GPU tasks. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

31% · raw 31 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “FrontierCode 2.0*” in the Tencent Hy4 release appendix, column “Hy4”. Harness unstated. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

78.8 score · raw 78.8 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Hy-CompanyBench 2.0 (internal)” in the Tencent Hy4 release appendix, column “Hy4”. GPT models in Codex 0.154.0, other models in Claude Code 2.1.234. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

45.4 score · raw 45.4 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Hy-SWE Max Verified 2.0 (internal)*” in the Tencent Hy4 release appendix, column “Hy4”. GPT models in Codex 0.154.0, other models in Claude Code 2.1.295. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

45.1 score · raw 45.1 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Hy-Backend 3.0 (internal)*” in the Tencent Hy4 release appendix, column “Hy4”. GPT models in Codex 0.154.0, other models in Claude Code 2.1.295. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

78.9 score · raw 78.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “WebCraftBench (internal)” in the Tencent Hy4 release appendix, column “Hy4”. GPT models in Codex 0.115.0, others in Claude Code 2.1.117. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

68.4 score · raw 68.4 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “GameFix Bench 2.0 (internal)” in the Tencent Hy4 release appendix, column “Hy4”. 100 tasks, hidden Playwright tests. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

37.1 score · raw 37.1 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “3DArena-Modeling (internal)” in the Tencent Hy4 release appendix, column “Hy4”. Blind pairwise Elo scaled to percentages. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

23.2 score · raw 23.2 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Goal2Repo (internal)*” in the Tencent Hy4 release appendix, column “Hy4”. avg@3. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

57.2 score · raw 57.2 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “AutoCatBench (internal)*” in the Tencent Hy4 release appendix, column “Hy4”. GPT models in Codex 0.115.0, others in Claude Code 2.1.117. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

60.2 score · raw 60.2 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Hy-LifeSearch (internal)” in the Tencent Hy4 release appendix, column “Hy4”. Internal search harness, LLM judge with per-task rubrics. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

59.9 score · raw 59.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Hy-BrowseComp 2.0 (internal)” in the Tencent Hy4 release appendix, column “Hy4”. Internal search harness, LLM judge against a unique reference answer. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

75.2% · raw 75.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “APEX-Agents v1.1 (pass@1)” in the Tencent Hy4 release appendix, column “Hy4”. Official Loop harness, up to 250 steps per trial, 8 CPU / 24 GB, separate verifier container. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

41% · raw 41 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Agents' Last Exam v1.1 (Pass Rate)*” in the Tencent Hy4 release appendix, column “Hy4”. Population and harness unstated. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

56.2% · raw 56.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “AutomationBench v1.0.6” in the Tencent Hy4 release appendix, column “Hy4”. All 600 public tasks released by Zapier, official method. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

70 score · raw 70 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “E-Bench 2.0 (internal)” in the Tencent Hy4 release appendix, column “Hy4”. MCP tools in a simple agent loop. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

79.9 score · raw 79.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Agents' First Exam-IF (internal)” in the Tencent Hy4 release appendix, column “Hy4”. 101 research tasks, rubric grading. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

48.8 score · raw 48.8 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Tag-bench (internal)” in the Tencent Hy4 release appendix, column “Hy4”. LLM judge plus programmatic verifiers. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

60.9 score · raw 60.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “PPTGen-Bench (internal)” in the Tencent Hy4 release appendix, column “Hy4”. GPT models in Codex 0.115.0, others in Claude Code 2.1.117. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

59.1% · raw 59.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “HLE-Full (with tools)” in the Tencent Hy4 release appendix, column “Hy4”. With tools; harness unstated. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

13 score · raw 13 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Hy-FrontierMath T5 (internal)*” in the Tencent Hy4 release appendix, column “Hy4”. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

5.5 score · raw 5.5 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “ProofBench (internal)*” in the Tencent Hy4 release appendix, column “Hy4”. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

19.8% · raw 19.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Terminal-Bench-Science 0.1” in the Tencent Hy4 release appendix, column “Hy4”. Claude Code harness, up to 500 turns and 13 hours per trial, 24 CPU / 48 GB. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

35.2 score · raw 35.2 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “PostTrainBench v1.2*” in the Tencent Hy4 release appendix, column “Hy4”. Claude Code 2.1.198 (OpenCode for Gemini), one H100, 10-hour budget. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

28.1 score · raw 28.1 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “Foundation Model Bench (internal)*” in the Tencent Hy4 release appendix, column “Hy4”. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

53.6 score · raw 53.6 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “TenureTrail (internal)*” in the Tencent Hy4 release appendix, column “Hy4”. GPT models in Codex 0.153.4, others in Claude Code 2.1.220. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

78.9% · raw 78.9 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “OSWorld 2.1 (Full, partial score)” in the Tencent Hy4 release appendix, column “Hy4”. Full suite, partial score. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

78.6% · raw 78.6 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “MMMU-Pro” in the Tencent Hy4 release appendix, column “Hy4”. Tool use not labelled; the Kimi K3.1 Pro value it copies (84.1) is Moonshot’s no-tools result. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

55.4% · raw 55.4 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “ZeroBench-main (with tools, pass@5)” in the Tencent Hy4 release appendix, column “Hy4”. Network-isolated Jupyter tool environment with image outputs, up to 80 tool rounds; 10 runs per problem to estimate pass@5. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

81 score · raw 81 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “BuddyVQA Atlas (with tools, Internal)” in the Tencent Hy4 release appendix, column “Hy4”. With tools. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

55.6 score · raw 55.6 score

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Exact label “BuddyVQA Atlas (without tools, Internal)” in the Tencent Hy4 release appendix, column “Hy4”. Without tools. Tencent reports each model at its highest available reasoning setting; no per-model effort label. Run date unpublished.

35 · raw 35 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Intelligence Index v5.0.1 (AA changelog, 3 November 2026: v5.0 with CritPt v2 regraded); never compared with v5.0 or v4.3.2 rows. AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public Intelligence Index evaluation page (title "Artificial Analysis Intelligence Index v5.0.1"), "Artificial Analysis Intelligence Index: Score" chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_index_score.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

78 k tokens/task · raw 78 k tokens/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Thousands of output tokens per task as displayed. Observed on the v5.0.1 page; bound to Intelligence Index v5.0.1 and never compared with v5.0 rows (five of the stored v5.0 cost and token values differ on the v5.0.1 page). AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public Intelligence Index v5.0.1 evaluation page, "Output Tokens per Task" stacked chart (reasoning plus answer total label), 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_index_tokens.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

39% · raw 39 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

CritPt v2 hard subset as regraded by AA on 3 November 2026; not the full set or the earlier CritPt (AA) rows. AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public CritPt v2 evaluation page, "Hard Subset" view, after AA's 3 November 2026 regrade, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_critpt_v2_hard.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

65.6% · raw 65.6 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public AutomationBench-AA evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_automationbench.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

1447 · raw 1447 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public AA-Briefcase evaluation page, combined Elo chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_briefcase_elo.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

1531 · raw 1531 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public AA-Briefcase evaluation page, "Analytical Quality & Presentation Elo" tab (Analytical Quality series), 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_briefcase_quality_presentation.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

1376 · raw 1376 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public AA-Briefcase evaluation page, "Analytical Quality & Presentation Elo" tab (Presentation series), 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_briefcase_quality_presentation.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

52.6% · raw 52.6 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public AA-Briefcase evaluation page, "Rubric Score (%)" tab, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_briefcase_rubric.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

5 · raw 5 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Signed knowledge index; v1.1 (3,000 questions) is not comparable with v1.0 rows. AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public AA-Omniscience evaluation page, "AA-Omniscience v1.1: Net Score" chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_omniscience.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

50% · raw 50 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Not comparable with v1.0 rows. AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public AA-Omniscience evaluation page, accuracy chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_omniscience.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

89% · raw 89 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Lower is better; not comparable with v1.0 rows. AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public AA-Omniscience evaluation page, hallucination rate chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_omniscience.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

12.9% · raw 12.9 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public Terminal-Bench-Science 0.1 evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_tbscience.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

31.4% · raw 31.4 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

AA harness; not Cognition board rows. AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public FrontierCode 2.0 evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_frontiercode2.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

32.7% · raw 32.7 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Not Humanity's Last Exam (AA). AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public HLE-Diamond evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_hle_diamond.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

21.6% · raw 21.6 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public GDP.pdf evaluation page, "All-pass Rate" tab, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_gdppdf.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

37.4% · raw 37.4 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

AA's "Tencent Hy4" (slug tencent-hy4) is the 4 November 2026 release: AA's model page says proprietary, released November 2026, text and image input, 1M context; not Hy4 preview. AA's label states no reasoning effort. Manual observation of Artificial Analysis's public Terminal-Bench 4.0 evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_tb4.txt). Configuration "Tencent Hy4". AA's displayed rounding; no API, download, hidden precision or inferred run date.

66.178% · raw 66.178 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.621653/task · raw 0.621653 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1058.904 s · raw 1058.904 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

74.211% · raw 74.211 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

58.611% · raw 58.611 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

56.829% · raw 56.829 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

56.642% · raw 56.642 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

60.114% · raw 60.114 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

68.024% · raw 68.024 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

77.040% · raw 77.04 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.922% · raw 86.922 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

57.679% · raw 57.679 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

58.050% · raw 58.05 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.195514/task · raw 1.195514 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2368.507 s · raw 2368.507 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

64.705% · raw 64.705 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

66.178% · raw 66.178 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

34.399% · raw 34.399 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.705% · raw 64.705 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.277206/task · raw 1.277206 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2238.942 s · raw 2238.942 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

52.579% · raw 52.579 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.513% · raw 63.513 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.804% · raw 82.804 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.815% · raw 64.815 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

53.253% · raw 53.253 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

38.887% · raw 38.887 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.632% · raw 64.632 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.409% · raw 86.409 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

80.437% · raw 80.437 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

47.591% · raw 47.591 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

12.500% · raw 12.5 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.036564/task · raw 1.036564 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1167.168 s · raw 1167.168 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.000% · raw 50 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.000% · raw 40 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.817% · raw 90.817 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

96.065% · raw 96.065 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.086% · raw 94.086 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.657% · raw 93.657 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

89.273% · raw 89.273 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.157% · raw 92.157 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.952% · raw 91.952 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.926% · raw 92.926 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

98.057% · raw 98.057 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.515% · raw 93.515 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

85.955% · raw 85.955 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

97.030% · raw 97.03 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.738% · raw 86.738 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.589% · raw 86.589 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.966% · raw 94.966 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

66.739% · raw 66.739 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

71.324% · raw 71.324 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.374% · raw 95.374 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.024% · raw 94.024 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.301% · raw 95.301 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

97.230% · raw 97.23 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.065% · raw 88.065 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.957% · raw 86.957 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.702% · raw 93.702 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.286% · raw 94.286 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

59.353% · raw 59.353 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.720395/task · raw 1.720395 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

3091.465 s · raw 3091.465 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

81.008% · raw 81.008 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.705% · raw 64.705 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

66.178% · raw 66.178 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

59.094% · raw 59.094 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Index Code Migration is the 60-task subset (50 CLI + 10 COBOL), not standalone Migration.

41.346% · raw 41.346 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

12.500% · raw 12.5 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

39.394% · raw 39.394 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

34.399% · raw 34.399 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

70.552% · raw 70.552 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.649017/task · raw 1.649017 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

3476.624 s · raw 3476.624 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

64.995% · raw 64.995 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

71.421% · raw 71.421 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.398% · raw 67.398 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

97.230% · raw 97.23 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

85.955% · raw 85.955 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.702% · raw 93.702 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

27.273% · raw 27.273 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.810% · raw 63.81 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.717% · raw 67.717 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

41.346% · raw 41.346 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.817% · raw 90.817 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

41.346% · raw 41.346 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$2.222804/task · raw 2.222804 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

5455.029 s · raw 5455.029 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

51.515% · raw 51.515 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

39.535% · raw 39.535 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

48.148% · raw 48.148 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

34.211% · raw 34.211 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

33.333% · raw 33.333 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

27.273% · raw 27.273 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.333% · raw 63.333 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

18.182% · raw 18.182 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.605% · raw 82.605 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.011% · raw 82.011 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.740% · raw 82.74 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

89.309% · raw 89.309 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

81.800% · raw 81.8 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.584% · raw 82.584 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.240% · raw 73.24 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.258% · raw 84.258 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

76.387% · raw 76.387 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

34.399% · raw 34.399 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.687683/task · raw 1.687683 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

3807.674 s · raw 3807.674 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

38.140% · raw 38.14 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

43.966% · raw 43.966 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

44.335% · raw 44.335 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

43.333% · raw 43.333 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

18.000% · raw 18 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

17.000% · raw 17 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.717% · raw 67.717 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

68.918% · raw 68.918 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

61.272% · raw 61.272 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

71.010% · raw 71.01 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

80.762% · raw 80.762 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

60.371% · raw 60.371 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.450% · raw 63.45 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

39.394% · raw 39.394 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.280113/task · raw 1.280113 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2600.367 s · raw 2600.367 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

51.852% · raw 51.852 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

35.714% · raw 35.714 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

36.364% · raw 36.364 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

37.037% · raw 37.037 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

46.667% · raw 46.667 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.000% · raw 50 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

81.008% · raw 81.008 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$2.394585/task · raw 2.394585 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2281.498 s · raw 2281.498 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

42 correct · raw 42 correct

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: Plotly Studio Embedded, one run on 43 private English data-analytics questions in a synthetic wind-energy warehouse. Claude Sonnet 5 judges against Plotly answer key. One or two questions can be noise; no error bars. Exact checkpoint hash, run date and effort unpublished. Company-operated analytics benchmark, distinct from provider results. Table row as displayed: “Hy4‡ Nov 4, 2026 42 / 43 $2.11 23.4M 1.9 min / 4.1 min 2 h 56 min”; ‡ marks open weights announced but not yet released. Cost is at OpenRouter prices on the run day; the date column is the release date Plotly lists. Serving precision unstated.