Examenos

49.1% · raw 49.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. Best effort; Cognition ran the evaluation (Claude Code harness), mean@5 overall score.

34.2% · raw 34.2 %

Alternative · low effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 209. Cognition run, Claude Code harness; effort sweep stated in text.

37.6% · raw 37.6 %

Alternative · medium effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 209. Cognition run, Claude Code harness; effort sweep stated in text.

45.3% · raw 45.3 %

Alternative · high effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 209. Cognition run, Claude Code harness; effort sweep stated in text.

45.7% · raw 45.7 %

Alternative · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 209. Cognition run; includes graded scores of 22 of 660 timed-out attempts (44.4% if counted as zero).

34.2% · raw 34.2 %

Alternative · low effort · Own vendor

Source/record date: 2026-11-09

Cognition primary public data.json (https://cognition.com/data/frontier-code-2.0/data.json), FrontierCode 2.0 main, 132 tasks; weighted new_score x100, not unweighted correct. Harness: claude-code. {"correct": 0.3742, "tokens": 23907, "cost": 2.623, "flagged_rate": 0.01515}. Cost is source-observed mean USD per rollout at list API pricing, not an API rate. Data checked 2026-11-10; evaluation date unpublished. Benchmark-owner board row; the headline is Anthropic's launch value (49.1%, xhigh), which matches this board within rounding.

37.6% · raw 37.59 %

Alternative · medium effort · Own vendor

Source/record date: 2026-11-09

Cognition primary public data.json (https://cognition.com/data/frontier-code-2.0/data.json), FrontierCode 2.0 main, 132 tasks; weighted new_score x100, not unweighted correct. Harness: claude-code. {"correct": 0.4106, "tokens": 28384, "cost": 3.103, "flagged_rate": 0.01667}. Cost is source-observed mean USD per rollout at list API pricing, not an API rate. Data checked 2026-11-10; evaluation date unpublished. Benchmark-owner board row; the headline is Anthropic's launch value (49.1%, xhigh), which matches this board within rounding.

45.3% · raw 45.29 %

Alternative · high effort · Own vendor

Source/record date: 2026-11-09

Cognition primary public data.json (https://cognition.com/data/frontier-code-2.0/data.json), FrontierCode 2.0 main, 132 tasks; weighted new_score x100, not unweighted correct. Harness: claude-code. {"correct": 0.4939, "tokens": 75978, "cost": 8.402, "flagged_rate": 0.0197}. Cost is source-observed mean USD per rollout at list API pricing, not an API rate. Data checked 2026-11-10; evaluation date unpublished. Benchmark-owner board row; the headline is Anthropic's launch value (49.1%, xhigh), which matches this board within rounding.

49.1% · raw 49.11 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-11-09

Cognition primary public data.json (https://cognition.com/data/frontier-code-2.0/data.json), FrontierCode 2.0 main, 132 tasks; weighted new_score x100, not unweighted correct. Harness: claude-code. {"correct": 0.5348, "tokens": 156009, "cost": 17.734, "flagged_rate": 0.01515}. Cost is source-observed mean USD per rollout at list API pricing, not an API rate. Data checked 2026-11-10; evaluation date unpublished. Benchmark-owner board row; the headline is Anthropic's launch value (49.1%, xhigh), which matches this board within rounding.

45.7% · raw 45.65 %

Alternative · max effort · Own vendor

Source/record date: 2026-11-09

Cognition primary public data.json (https://cognition.com/data/frontier-code-2.0/data.json), FrontierCode 2.0 main, 132 tasks; weighted new_score x100, not unweighted correct. Harness: claude-code. {"correct": 0.5, "tokens": 417808, "cost": 55.868, "flagged_rate": 0.03939}. Cost is source-observed mean USD per rollout at list API pricing, not an API rate. Data checked 2026-11-10; evaluation date unpublished. Benchmark-owner board row; the headline is Anthropic's launch value (49.1%, xhigh), which matches this board within rounding.

69.9% · raw 69.9 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. Proximal agent harness at max effort, mean of five trials per task (card p. 212).

51.4% · raw 51.4 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. Zapier private AutomationBench 1.0.6 set (card p. 240-241), Zapier run of 8 October 2026 with the API default fallbacks.

39.7% · raw 39.7 %

Alternative · low effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 241. Zapier private 1.0.6 set, low effort.

74.6% · raw 74.6 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. Claude Code --bare at max effort, no internet access, 10 trials per task; safeguards on (fallback on 1.3% of requests). Mythos 5.2 without safeguards: 75.3%, not stored here.

89.3% · raw 89.3 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. Official 82-task offline set, partial score, Pass@1 over five attempts, 1080p, 500 steps, Opus 4.8 grader (card p. 235).

54.6% · raw 54.6 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 235. Official 82-task offline set, strict pass rate, five attempts per task.

1785 · raw 1785 Elo

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. Elo; evaluations run independently by Artificial Analysis (card p. 239). Rubric pass rate 66.1% is not stored (no rubric definition).

1738 · raw 1738 Elo

Alternative · xhigh effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 239. Elo at xhigh effort, run by Artificial Analysis.

41.3% · raw 41.3 %

Headline · high effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. Scale AI ran Claude models at default settings: high effort, 200K context, 32K tokens per response; 99 of 240 projects (card p. 242).

74.2% · raw 74.2 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. Web search, web fetch, programmatic tool calling and code execution; 980K-token budget; Opus 4.6 grader; HLE-discussing sources blocklisted (card p. 216).

73.2% · raw 73.2 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 207. Table 8.1.A, reasoning only, no tools; Opus 4.6 grader.

73.4% · raw 73.4 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. No tools; Gemini 3.5 Flash grader as on the Surge AI leaderboard; mean of five runs (card p. 229).

91.6% · raw 91.6 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 229. Container with the image and an image-cropping tool; Gemini 3.5 Flash grader; mean of five runs.

64.9% · raw 64.9 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 launch table (9 November 2026); safeguards on, highest available effort unless noted. Anthropic internal run, highest available effort (max).

93% · raw 93 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 208. Average of five trials; Table 8.1.A footnote: max effort, default sampling.

96.6% · raw 96.6 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 208. Average of five trials.

74.1% · raw 74.1 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 208. Average of five trials.

74.3% · raw 74.3 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 208. 113 tasks, average over five trials; harness not named.

67.6% · raw 67.6 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 210. Claude Code --bare, max effort, no internet access (cached resources, frozen search index), 10 trials per task; safeguards on (fallback on 2.5% of requests).

60% · raw 60 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 213. Cursor measured and reported in its production harness; Cursor attributes part of the score to prompt and harness improvements, so it may not compare directly with earlier cards.

56.3% · raw 56.3 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 213. Cursor-reported.

56.9% · raw 56.9 %

Alternative · high effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 213. Cursor-reported.

54.8% · raw 54.8 %

Alternative · medium effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 213. Cursor-reported.

53.2% · raw 53.2 %

Alternative · low effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 213. Cursor-reported.

85.6% · raw 85.6 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 216. mini-swe-agent, no six-hour limit; ran out of context before submitting on 68% of tasks. With remaining-context counts shown it scored 91.6% (not stored).

75.4% · raw 75.4 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 233. Mean of five runs.

83.5% · raw 83.5 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 233. Mean of five runs.

83.4% · raw 83.4 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 237. Agentic, documents preselected as extracted text; mean of five runs.

72.9% · raw 72.9 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 237. 133-question subset; mean of five runs.

7.2% · raw 7.2 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 238. Artificial Analysis Harvey LAB-AA v1.1 held-out set, hallucination-gated all-pass rate (10.8% before the hallucination check).

1831 · raw 1831 Elo

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 239. Elo anchored to DeepSeek-V4.1-Flash (max) at 1600; run independently by Artificial Analysis.

1751 · raw 1751 Elo

Alternative · xhigh effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 239. Elo at xhigh, run by Artificial Analysis.

64.4% · raw 64.4 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 247. 100 EHR tasks, five attempts each; Opus 5 grader.

55% · raw 55 %

Alternative · low effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 247. Three runs pooled.

61.8% · raw 61.8 %

Alternative · high effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 247.

64.6% · raw 64.6 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 247.

96% · raw 96 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 248. Average over 42 languages, single trial, safety classifiers on; blocked or unanswered questions (0.5%) left out of the average.

94.1% · raw 94.1 %

Headline · max effort · Own vendor

Source/record date: 2026-11-09

Anthropic Claude Fable 5.2 and Mythos 5.2 system card, p. 248. Average over 11 languages, five trials, safety classifiers on.

77.377% · raw 77.377 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$6.672375/task · raw 6.672375 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

979.091 s · raw 979.091 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

81.388% · raw 81.388 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.625% · raw 73.625 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.183% · raw 73.183 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

68.524% · raw 68.524 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

81.934% · raw 81.934 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

76.968% · raw 76.968 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.252% · raw 82.252 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

87.733% · raw 87.733 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

71.047% · raw 71.047 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.583% · raw 63.583 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$4.938442/task · raw 4.938442 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

495.484 s · raw 495.484 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

69.470% · raw 69.47 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

77.377% · raw 77.377 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

32.092% · raw 32.092 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

69.470% · raw 69.47 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$4.833915/task · raw 4.833915 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

207.619 s · raw 207.619 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

57.595% · raw 57.595 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

75.546% · raw 75.546 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.012% · raw 86.012 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

74.256% · raw 74.256 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

61.884% · raw 61.884 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

43.385% · raw 43.385 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

65.386% · raw 65.386 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.103% · raw 90.103 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

76.850% · raw 76.85 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

51.807% · raw 51.807 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

5.000% · raw 5 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$19.366751/task · raw 19.366751 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1954.199 s · raw 1954.199 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

30.000% · raw 30 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.458% · raw 88.458 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.051% · raw 86.051 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.039% · raw 91.039 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.112% · raw 90.112 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

87.889% · raw 87.889 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.980% · raw 90.98 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

76.370% · raw 76.37 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.801% · raw 91.801 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.053% · raw 95.053 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.321% · raw 92.321 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.457% · raw 84.457 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.914% · raw 91.914 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

81.004% · raw 81.004 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

85.569% · raw 85.569 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.772% · raw 90.772 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.182% · raw 88.182 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.625% · raw 90.625 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.374% · raw 95.374 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.861% · raw 84.861 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.910% · raw 88.91 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.003% · raw 93.003 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.613% · raw 86.613 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

81.940% · raw 81.94 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.458% · raw 90.458 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

87.374% · raw 87.374 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.982% · raw 73.982 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.638668/task · raw 0.638668 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

135.792 s · raw 135.792 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

26.718% · raw 26.718 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

55.034% · raw 55.034 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

70.280% · raw 70.28 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

71.309% · raw 71.309 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$17.939202/task · raw 17.939202 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2621.822 s · raw 2621.822 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

93.771% · raw 93.771 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

69.470% · raw 69.47 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

77.377% · raw 77.377 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

85.848% · raw 85.848 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Index Code Migration is the 60-task subset (50 CLI + 10 COBOL), not standalone Migration.

46.154% · raw 46.154 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

5.000% · raw 5 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

65.152% · raw 65.152 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

32.092% · raw 32.092 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

69.175% · raw 69.175 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$8.877409/task · raw 8.877409 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

890.047 s · raw 890.047 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

65.188% · raw 65.188 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

65.358% · raw 65.358 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

71.604% · raw 71.604 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.003% · raw 93.003 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.457% · raw 84.457 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.458% · raw 90.458 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

18.182% · raw 18.182 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

66.494% · raw 66.494 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.008% · raw 67.008 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

46.154% · raw 46.154 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.458% · raw 88.458 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

46.154% · raw 46.154 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$3.956441/task · raw 3.956441 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

416.198 s · raw 416.198 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

59.091% · raw 59.091 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

41.860% · raw 41.86 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.741% · raw 40.741 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

39.474% · raw 39.474 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

45.614% · raw 45.614 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

18.182% · raw 18.182 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

80.000% · raw 80 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

27.273% · raw 27.273 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.782% · raw 86.782 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

89.786% · raw 89.786 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.639% · raw 86.639 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.020% · raw 91.02 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

87.463% · raw 87.463 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.688% · raw 86.688 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

75.691% · raw 75.691 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.114% · raw 95.114 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.763% · raw 83.763 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.265% · raw 83.265 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$69.18581/task · raw 69.18581 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

11202.435 s · raw 11202.435 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

81.020% · raw 81.02 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

90.000% · raw 90 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

4.090 points/5 · raw 4.09 points/5

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.740% · raw 63.74 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

2080.932 s · raw 2080.932 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

83.704% · raw 83.704 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

51.802% · raw 51.802 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

55.714% · raw 55.714 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

32.092% · raw 32.092 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$3.309035/task · raw 3.309035 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

299.743 s · raw 299.743 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

27.907% · raw 27.907 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

46.667% · raw 46.667 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

36.512% · raw 36.512 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.000% · raw 50 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

13.278% · raw 13.278 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.008% · raw 67.008 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.481% · raw 64.481 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

68.032% · raw 68.032 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

65.974% · raw 65.974 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.365% · raw 86.365 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

60.091% · raw 60.091 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

58.203% · raw 58.203 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

65.152% · raw 65.152 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$14.564414/task · raw 14.564414 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

3307.186 s · raw 3307.186 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

55.556% · raw 55.556 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

59.524% · raw 59.524 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

81.818% · raw 81.818 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

48.148% · raw 48.148 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.333% · raw 73.333 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.333% · raw 93.333 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

75.000% · raw 75 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.771% · raw 93.771 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$22.13226/task · raw 22.13226 USD/task

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2437.279 s · raw 2437.279 s

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

97.5% · raw 97.5 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix.

98.5% · raw 98.5 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Headline: best verified value; source matrix effort order breaks ties.

97.5% · raw 97.5 %

Alternative · high effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix.

97.5% · raw 97.5 %

Alternative · medium effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix.

94.5% · raw 94.5 %

Alternative · low effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix.

92.5% · raw 92.5 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix.

93.8% · raw 93.8 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Headline: best verified value; source matrix effort order breaks ties.

90.4% · raw 90.4 %

Alternative · high effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix.

89.6% · raw 89.6 %

Alternative · medium effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix.

86.7% · raw 86.7 %

Alternative · low effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix.

65.7% · raw 65.66 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Standard (notes carry). ARC note: At max reasoning with the Standard harness, 28 of the 55 semi-private games ran for the full 12-hour limit, which capped how far testing could progress in those games.

81.2% · raw 81.17 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Standard (notes carry). Headline: best verified value; source matrix effort order breaks ties.

79.3% · raw 79.26 %

Alternative · high effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Standard (notes carry).

73.3% · raw 73.32 %

Alternative · medium effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Standard (notes carry).

52% · raw 52 %

Alternative · low effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Standard (notes carry).

94.9% · raw 94.9 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Provider Adapter.

97.3% · raw 97.27 %

Alternative · xhigh effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Provider Adapter.

97.7% · raw 97.72 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Provider Adapter. Headline: best verified value; source matrix effort order breaks ties.

96.8% · raw 96.79 %

Alternative · medium effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Provider Adapter.

92.3% · raw 92.31 %

Alternative · low effort · Independent evaluator

Source/record date: 2026-11-10

Semi-Private verified matrix. Harness: Provider Adapter.

16.4% · raw 16.4 %

Headline · max effort · Independent evaluator

Source/record date: 2026-11-10

Harness: Claude Code (board “Show harness” column), effort label MAX. GDPx v1.0: 206 tasks from 14 partners across 11 domains, domains weighted equally and partners equally within a domain; 8-hour default time limit; mean of 3 attempts. Board rank 1 of 21; run dates unpublished. Score uses each task’s intended grading (binary or continuous by domain).

$88.15/task · raw 88.15 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-11-10

Harness: Claude Code (board “Show harness” column), effort label MAX. GDPx v1.0: 206 tasks from 14 partners across 11 domains, domains weighted equally and partners equally within a domain; 8-hour default time limit; mean of 3 attempts. Board rank 1 of 21; run dates unpublished. Average cost per task as displayed.

1230000 tokens/task · raw 1230000 tokens/task

Headline · max effort · Independent evaluator

Source/record date: 2026-11-10

Harness: Claude Code (board “Show harness” column), effort label MAX. GDPx v1.0: 206 tasks from 14 partners across 11 domains, domains weighted equally and partners equally within a domain; 8-hour default time limit; mean of 3 attempts. Board rank 1 of 21; run dates unpublished. Average output tokens per task, displayed as 1.23M.