Examenos

54.3% · raw 54.3 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Internal seven-suite composite; official Meta PSI, not an independent index. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

10% · raw 10 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Vendor cites AA v1.1 hallucination-gated all-pass rate. Direct AA Harvey board checked separately; new Wonder configurations not yet in its 25-row table. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

56.6% · raw 56.6 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Vendor cites Vals private-test-set accuracy; independently checked Vals values retain full published precision in evaluators.json. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

60.5% · raw 60.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; 525 professional health questions; no tools, GPT-5.4 rubric grader, length-normalized weighted rubric score. Claude scores cited from their Anthropic system cards. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

39.1% · raw 39.1 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Vendor-cited strict pass@1: every deterministic end-state assertion must pass. Not AA model-page partial reward (68.09/71.33 for Wonder/Full). Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

30.8% · raw 30.8 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Vendor cites Surge professional PDF work score. Direct AA GDP.PDF all-pass for Full is 32.6, not the vendor-cited 33.2; keep separate. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

70.5% · raw 70.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; 65 professional tasks across 35 occupations; mean rubric reward. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

72.6% · raw 72.6 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Updated V2.1 108-task GUI workflows; common internal framework preserving task environments/graders. Opus is Anthropic provider-reported partial reward. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

34.3% · raw 34.3 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Strict binary completion; Opus binary result absent, not zero. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

85.6% · raw 85.6 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Internal weighted rubric recall; 64K–100K-token multi-turn histories with distractors, conflicts and updates. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

52% · raw 52 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Meta quotes AA reported accuracy; independent exact AA observations remain in evaluators.json. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

78.8% · raw 78.8 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; 113 tasks, 91 repositories, five languages, pass@1. Meta attributes Wonder/Full/Opus to Datacurve; GPT to OpenAI and Fable to Anthropic. Current public Datacurve board predates Wonder; retain vendor citation. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

53.1% · raw 53.1 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Abundant AI pass@1; GPT uses Codex and Fable Claude Code, so not a common-harness comparison. Full effort xhigh. Do not conflate with SWE-Marathon v1.1 partial-reward records. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

50% · raw 50 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; mini-swe-agent, internet disabled after setup, rollouts passing at least 95% of hidden tests; Almost Solved, not fully resolved. Exact task/revision details unpublished. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

52 · raw 52 index

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Rounded vendor citation of AA Intelligence Index v4.3; exact independent index values kept separately. Source checked 12 October 2026; evaluation date unpublished. Transcribed from model-card page 3 image; max effort except Full SWE-Marathon xhigh.

44.9% · raw 44.9 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Meta internal personal-agent evaluation of persistent work through blockers, multiple interactions and dynamically changing environments, including housing, travel, IT and finance. Source checked 12 October 2026; evaluation date unpublished. Figure/table transcribed visually; do not infer unpublished configuration details.

94.5% · raw 94.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Meta announcement figure: compliance with defensive cyber requests. Higher is better; dataset, denominator, evaluation dates and detailed harness unpublished. Source checked 12 October 2026; evaluation date unpublished. Figure/table transcribed visually; do not infer unpublished configuration details.

93.7% · raw 93.7 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Meta announcement figure: refusal of cyber misuse requests. Higher is better; dataset, denominator and detailed harness unpublished. Source checked 12 October 2026; evaluation date unpublished. Figure/table transcribed visually; do not infer unpublished configuration details.

98% · raw 98 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Meta launch figure BioTIER refusal rate. Language condition unspecified in the chart; retain separately from model-card English/multilingual/adversarial conditions. Source checked 12 October 2026; evaluation date unpublished. Figure/table transcribed visually; do not infer unpublished configuration details.

3.4% · raw 3.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Meta launch figure: rate of mishandling personal information; lower is better. Dataset, denominator and evaluation dates unpublished. Source checked 12 October 2026; evaluation date unpublished. Figure/table transcribed visually; do not infer unpublished configuration details.

10.5% · raw 10.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-12

Harness: Meta launch evaluation; Meta launch figure: unauthorized code execution attack success rate; lower is better. Exact prompt-injection harness and denominator unpublished. Source checked 12 October 2026; evaluation date unpublished. Figure/table transcribed visually; do not infer unpublished configuration details.

64.877% · raw 64.877 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2123.575,"stderr":1.067,"cost_per_test":10.647504,"token_totals":{"input_tokens":18224226557,"output_tokens":97195877,"reasoning_tokens":52653929,"cache_read_tokens":17645554776,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

$10.647504/task · raw 10.647504 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2123.575,"stderr":1.067,"cost_per_test":10.647504,"token_totals":{"input_tokens":18224226557,"output_tokens":97195877,"reasoning_tokens":52653929,"cache_read_tokens":17645554776,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

2123.575 s · raw 2123.575 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2123.575,"stderr":1.067,"cost_per_test":10.647504,"token_totals":{"input_tokens":18224226557,"output_tokens":97195877,"reasoning_tokens":52653929,"cache_read_tokens":17645554776,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

83.089% · raw 83.089 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":350.076,"stderr":2.49,"cost_per_test":1.802723,"token_totals":{"input_tokens":144518172,"output_tokens":1830347,"reasoning_tokens":114242,"cache_read_tokens":138037760,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":"OpenHands"}

65.545% · raw 65.545 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":310.129,"stderr":2.031,"cost_per_test":2.620705,"token_totals":{"input_tokens":398379578,"output_tokens":15303665,"reasoning_tokens":9870222,"cache_read_tokens":190235588,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

72.503% · raw 72.503 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

69.781% · raw 69.781 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":9687.179,"stderr":4.035,"cost_per_test":47.552905,"token_totals":{"input_tokens":11958750391,"output_tokens":23001800,"reasoning_tokens":10801141,"cache_read_tokens":11889714692,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

50.481% · raw 50.481 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":927.111,"stderr":3.475,"cost_per_test":3.980242,"token_totals":{"input_tokens":732491974,"output_tokens":17268053,"reasoning_tokens":11668590,"cache_read_tokens":644062808,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

23.750% · raw 23.75 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":906.14,"stderr":3.646,"cost_per_test":7.081987,"token_totals":{"input_tokens":1414833931,"output_tokens":12111113,"reasoning_tokens":1875186,"cache_read_tokens":1374979944,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

49.495% · raw 49.495 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3280.687,"stderr":2.201,"cost_per_test":14.657694,"token_totals":{"input_tokens":2276758314,"output_tokens":10781411,"reasoning_tokens":7812084,"cache_read_tokens":2198750960,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

36.269% · raw 36.269 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":492.843,"stderr":3.47,"cost_per_test":1.884439,"token_totals":{"input_tokens":225535207,"output_tokens":8770631,"reasoning_tokens":5466021,"cache_read_tokens":187857044,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

65.545% · raw 65.545 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":310.129,"stderr":2.031,"cost_per_test":2.620705,"token_totals":{"input_tokens":398379578,"output_tokens":15303665,"reasoning_tokens":9870222,"cache_read_tokens":190235588,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

$2.620705/task · raw 2.620705 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":310.129,"stderr":2.031,"cost_per_test":2.620705,"token_totals":{"input_tokens":398379578,"output_tokens":15303665,"reasoning_tokens":9870222,"cache_read_tokens":190235588,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

310.129 s · raw 310.129 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":310.129,"stderr":2.031,"cost_per_test":2.620705,"token_totals":{"input_tokens":398379578,"output_tokens":15303665,"reasoning_tokens":9870222,"cache_read_tokens":190235588,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

53.102% · raw 53.102 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":310.129,"stderr":2.212,"cost_per_test":2.620705,"token_totals":{"input_tokens":398379578,"output_tokens":15303665,"reasoning_tokens":9870222,"cache_read_tokens":190235588,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

67.507% · raw 67.507 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":403.948,"stderr":6.097,"cost_per_test":3.322322,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

82.207% · raw 82.207 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":241.751,"stderr":4.852,"cost_per_test":2.384636,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

67.685% · raw 67.685 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":256.023,"stderr":6.696,"cost_per_test":1.433767,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

54.967% · raw 54.967 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":412.873,"stderr":6.611,"cost_per_test":3.823525,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

45.558% · raw 45.558 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":451.591,"stderr":7.199,"cost_per_test":4.028488,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

64.735% · raw 64.735 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":292.846,"stderr":6.669,"cost_per_test":2.505207,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

90.331% · raw 90.331 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":161.195,"stderr":3.884,"cost_per_test":0.855159,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

78.189% · raw 78.189 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":349.46,"stderr":5.455,"cost_per_test":3.355491,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

38.730% · raw 38.73 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":264.44,"stderr":6.612,"cost_per_test":2.078374,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

72.503% · raw 72.503 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

$5.599337/task · raw 5.599337 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

1034.437 s · raw 1034.437 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

76.345% · raw 76.345 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1119.036,"stderr":3.104,"cost_per_test":7.413135,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

68.848% · raw 68.848 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":949.838,"stderr":2.723,"cost_per_test":3.785539,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

64.223% · raw 64.223 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1638.99,"stderr":5.103,"cost_per_test":9.500848,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

63.666% · raw 63.666 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":968.128,"stderr":7.151,"cost_per_test":5.11822,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

80.373% · raw 80.373 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":857.035,"stderr":3.445,"cost_per_test":4.182315,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

70.638% · raw 70.638 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":917.117,"stderr":5.175,"cost_per_test":4.824542,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

77.902% · raw 77.902 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":996.365,"stderr":4.38,"cost_per_test":6.189429,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

87.732% · raw 87.732 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":835.941,"stderr":3.186,"cost_per_test":5.374192,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

62.987% · raw 62.987 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":951.464,"stderr":6.965,"cost_per_test":3.120758,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

70.042% · raw 70.042 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

58.403% · raw 58.403 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

65.000% · raw 65 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

62.332% · raw 62.332 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

81.521% · raw 81.521 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

79.224% · raw 79.224 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

75.487% · raw 75.487 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

65.789% · raw 65.789 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

86.381% · raw 86.381 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

69.423% · raw 69.423 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

91.040% · raw 91.04 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

84.424% · raw 84.424 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

62.565% · raw 62.565 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

63.408% · raw 63.408 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

61.361% · raw 61.361 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

86.787% · raw 86.787 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

75.307% · raw 75.307 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1034.437,"stderr":2.414,"cost_per_test":5.599337,"token_totals":{"input_tokens":1072958990,"output_tokens":8128857,"reasoning_tokens":5046443,"cache_read_tokens":1021915980,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

72.355% · raw 72.355 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":9713.56,"stderr":3.187,"cost_per_test":47.931802,"token_totals":{"input_tokens":28436080448,"output_tokens":55673508,"reasoning_tokens":25796839,"cache_read_tokens":28273954572,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

$47.931802/task · raw 47.931802 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":9713.56,"stderr":3.187,"cost_per_test":47.931802,"token_totals":{"input_tokens":28436080448,"output_tokens":55673508,"reasoning_tokens":25796839,"cache_read_tokens":28273954572,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

9713.560 s · raw 9713.56 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":9713.56,"stderr":3.187,"cost_per_test":47.931802,"token_totals":{"input_tokens":28436080448,"output_tokens":55673508,"reasoning_tokens":25796839,"cache_read_tokens":28273954572,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

67.140% · raw 67.14 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":11847.177,"stderr":2.69,"cost_per_test":61.105472,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

88.000% · raw 88 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3312.708,"stderr":9.866,"cost_per_test":8.410792,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

2.975 points/5 · raw 2.975 points/5

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":9713.56,"stderr":0.043,"cost_per_test":47.931802,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

23.750% · raw 23.75 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":906.14,"stderr":3.646,"cost_per_test":7.081987,"token_totals":{"input_tokens":1414833931,"output_tokens":12111113,"reasoning_tokens":1875186,"cache_read_tokens":1374979944,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

$7.081987/task · raw 7.081987 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":906.14,"stderr":3.646,"cost_per_test":7.081987,"token_totals":{"input_tokens":1414833931,"output_tokens":12111113,"reasoning_tokens":1875186,"cache_read_tokens":1374979944,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

906.140 s · raw 906.14 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":906.14,"stderr":3.646,"cost_per_test":7.081987,"token_totals":{"input_tokens":1414833931,"output_tokens":12111113,"reasoning_tokens":1875186,"cache_read_tokens":1374979944,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1023.55,"stderr":20.0,"cost_per_test":7.544454,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

10.000% · raw 10 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":842.758,"stderr":10.0,"cost_per_test":6.853377,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

30.000% · raw 30 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":936.4,"stderr":20.0,"cost_per_test":6.016663,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":945.124,"stderr":20.0,"cost_per_test":7.317233,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

30.000% · raw 30 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":749.014,"stderr":20.0,"cost_per_test":7.059417,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

30.000% · raw 30 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":869.038,"stderr":20.0,"cost_per_test":5.385755,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

40.000% · raw 40 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":894.054,"stderr":24.495,"cost_per_test":7.82908,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

10.000% · raw 10 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":830.088,"stderr":10.0,"cost_per_test":6.429998,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

40.000% · raw 40 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":972.262,"stderr":24.495,"cost_per_test":7.098804,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":862.152,"stderr":0.0,"cost_per_test":7.283077,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

70.000% · raw 70 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1067.054,"stderr":20.0,"cost_per_test":11.537685,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":820.33,"stderr":20.0,"cost_per_test":5.178019,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

10.000% · raw 10 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1090.864,"stderr":10.0,"cost_per_test":12.604977,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":975.282,"stderr":20.0,"cost_per_test":5.625299,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

40.000% · raw 40 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":622.17,"stderr":24.495,"cost_per_test":3.50291,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

30.000% · raw 30 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":921.1,"stderr":20.0,"cost_per_test":7.517888,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":818.0,"stderr":20.0,"cost_per_test":7.383931,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

30.000% · raw 30 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":902.572,"stderr":20.0,"cost_per_test":3.602648,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

10.000% · raw 10 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":850.952,"stderr":10.0,"cost_per_test":10.904622,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

30.000% · raw 30 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":790.378,"stderr":20.0,"cost_per_test":5.727136,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":969.446,"stderr":20.0,"cost_per_test":7.587681,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":873.104,"stderr":0.0,"cost_per_test":5.425917,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":920.036,"stderr":20.0,"cost_per_test":7.713584,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1201.638,"stderr":20.0,"cost_per_test":6.121717,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

96.010% · raw 96.01 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":906.14,"stderr":0.236,"cost_per_test":7.081987,"token_totals":{"input_tokens":1414833931,"output_tokens":12111113,"reasoning_tokens":1875186,"cache_read_tokens":1374979944,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

95.652% · raw 95.652 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1023.55,"stderr":1.23,"cost_per_test":7.544454,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

95.878% · raw 95.878 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":842.758,"stderr":1.192,"cost_per_test":6.853377,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

96.455% · raw 96.455 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":936.4,"stderr":1.132,"cost_per_test":6.016663,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

94.464% · raw 94.464 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":945.124,"stderr":1.348,"cost_per_test":7.317233,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

96.275% · raw 96.275 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":749.014,"stderr":1.188,"cost_per_test":7.059417,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

97.603% · raw 97.603 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":869.038,"stderr":0.897,"cost_per_test":5.385755,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

97.910% · raw 97.91 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":894.054,"stderr":0.812,"cost_per_test":7.82908,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

97.703% · raw 97.703 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":830.088,"stderr":0.892,"cost_per_test":6.429998,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

97.440% · raw 97.44 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":972.262,"stderr":0.924,"cost_per_test":7.098804,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

93.071% · raw 93.071 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":862.152,"stderr":1.557,"cost_per_test":7.283077,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

98.515% · raw 98.515 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1067.054,"stderr":0.696,"cost_per_test":11.537685,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

94.803% · raw 94.803 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":820.33,"stderr":1.331,"cost_per_test":5.178019,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

95.044% · raw 95.044 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1090.864,"stderr":1.174,"cost_per_test":12.604977,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

95.134% · raw 95.134 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":975.282,"stderr":1.248,"cost_per_test":5.625299,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

96.182% · raw 96.182 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":622.17,"stderr":1.158,"cost_per_test":3.50291,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

96.140% · raw 96.14 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":921.1,"stderr":1.17,"cost_per_test":7.517888,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

97.153% · raw 97.153 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":818.0,"stderr":0.994,"cost_per_test":7.383931,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

96.414% · raw 96.414 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":902.572,"stderr":1.176,"cost_per_test":3.602648,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

95.113% · raw 95.113 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":850.952,"stderr":1.324,"cost_per_test":10.904622,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

97.230% · raw 97.23 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":790.378,"stderr":0.887,"cost_per_test":5.727136,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

94.194% · raw 94.194 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":969.446,"stderr":1.33,"cost_per_test":7.587681,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

92.475% · raw 92.475 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":873.104,"stderr":1.528,"cost_per_test":5.425917,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

96.183% · raw 96.183 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":920.036,"stderr":1.186,"cost_per_test":7.713584,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

96.970% · raw 96.97 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1201.638,"stderr":0.996,"cost_per_test":6.121717,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

$1.884439/task · raw 1.884439 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":492.843,"stderr":3.47,"cost_per_test":1.884439,"token_totals":{"input_tokens":225535207,"output_tokens":8770631,"reasoning_tokens":5466021,"cache_read_tokens":187857044,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

492.843 s · raw 492.843 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":492.843,"stderr":3.47,"cost_per_test":1.884439,"token_totals":{"input_tokens":225535207,"output_tokens":8770631,"reasoning_tokens":5466021,"cache_read_tokens":187857044,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

39.535% · raw 39.535 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":504.76,"stderr":7.544,"cost_per_test":1.775974,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

43.333% · raw 43.333 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":446.798,"stderr":9.202,"cost_per_test":1.712984,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

50.000% · raw 50 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":653.458,"stderr":9.285,"cost_per_test":2.232327,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

40.000% · raw 40 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":206.578,"stderr":9.097,"cost_per_test":0.792122,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

23.333% · raw 23.333 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":551.297,"stderr":7.854,"cost_per_test":2.329763,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":604.74,"stderr":7.428,"cost_per_test":2.510462,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

67.563% · raw 67.563 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":504.76,"stderr":6.892,"cost_per_test":1.775974,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

67.655% · raw 67.655 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":446.798,"stderr":8.257,"cost_per_test":1.712984,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

77.655% · raw 77.655 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":653.458,"stderr":7.258,"cost_per_test":2.232327,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

74.438% · raw 74.438 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":206.578,"stderr":7.066,"cost_per_test":0.792122,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

69.850% · raw 69.85 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":551.297,"stderr":7.875,"cost_per_test":2.329763,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

71.362% · raw 71.362 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":604.74,"stderr":7.408,"cost_per_test":2.510462,"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

83.089% · raw 83.089 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. Independent model-plus-harness web-application evaluation. Preserve OpenHands versus other agent variants; source costs/runtime are task observations, not API tariffs. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":350.076,"stderr":2.49,"cost_per_test":1.802723,"token_totals":{"input_tokens":144518172,"output_tokens":1830347,"reasoning_tokens":114242,"cache_read_tokens":138037760,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":"OpenHands"}

83.090% · raw 83.09 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±2.49 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained. Demoted 2026-10-20: current publisher observation from https://www.vals.ai/benchmarks/vibe-code is the headline for this same version/configuration family; this historical value and source remain available. Revised benchmark versions use separate metrics.

$1.802723/task · raw 1.802723 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. Independent model-plus-harness web-application evaluation. Preserve OpenHands versus other agent variants; source costs/runtime are task observations, not API tariffs. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":350.076,"stderr":2.49,"cost_per_test":1.802723,"token_totals":{"input_tokens":144518172,"output_tokens":1830347,"reasoning_tokens":114242,"cache_read_tokens":138037760,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":"OpenHands"} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

$1.8/task · raw 1.8 USD/task

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-12

Published evaluator Cost / Test; max default, suite-specific harness. Vibe Code uses OpenHands. Not token prices or subscription charges; Full priced evaluator run does not establish a public API. Demoted 2026-10-20: current publisher observation from https://www.vals.ai/benchmarks/vibe-code is the headline for this same version/configuration family; this historical value and source remain available. Revised benchmark versions use separate metrics.

350.076 s · raw 350.076 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. Independent model-plus-harness web-application evaluation. Preserve OpenHands versus other agent variants; source costs/runtime are task observations, not API tariffs. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":350.076,"stderr":2.49,"cost_per_test":1.802723,"token_totals":{"input_tokens":144518172,"output_tokens":1830347,"reasoning_tokens":114242,"cache_read_tokens":138037760,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":"OpenHands"} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

4.75 acronyms/response · raw 4.75 acronyms/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Acronyms: Two or more capital letters in a row (AI, USA) or dotted forms (U.S.). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 4.75.

14.30 words/sentence · raw 14.3 words/sentence

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Avg. sentence length: Words divided by sentences. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 14.3.

9.08 bold spans/response · raw 9.08 bold spans/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Bold text: Uses of bold: each unbroken run of bold text counts once, so a single bolded word and a fully bolded sentence both count as 1. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 9.08.

4362 characters/response · raw 4362 characters/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Characters: Characters in the prose, spaces included, after code and URLs are removed. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 4,362.

12.60 colons/response · raw 12.6 colons/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Colons: Colons. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 12.6.

19.10 contractions/response · raw 19.1 contractions/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Contractions: Contractions such as “don’t”, “you’re” and “it’s”; possessives such as “John’s” do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 19.1.

0.33 ellipses/response · raw 0.33 ellipses/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Ellipses: Three periods (...) or the ellipsis character (…). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.33.

6.87 em dashes/response · raw 6.87 em dashes/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Em dashes: The em dash character (—); double hyphens do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 6.87.

0.01 emojis/response · raw 0.01 emojis/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Emojis: Each emoji as the reader sees it. An emoji built from several characters, such as a family (👨‍👩‍👧), still counts as one emoji. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.01.

15% · raw 15 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Ends with a question: Whether the answer’s last character is a question mark, reported as the share of responses that do. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 15%.

0.18 exclamations/response · raw 0.18 exclamations/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Exclamations: Exclamation marks. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.18.

4.03 headings/response · raw 4.03 headings/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Headings: Headings of any level. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 4.03.

4.94 italic spans/response · raw 4.94 italic spans/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Italics: Uses of italics: each unbroken run of italic text counts once, so a single italicized word and a fully italicized sentence both count as 1. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 4.94.

2.39 lists/response · raw 2.39 lists/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Lists: Top-level bulleted or numbered lists; nested sub-lists do not count separately. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 2.39.

19.80 paragraphs/response · raw 19.8 paragraphs/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Paragraphs: Paragraphs of prose; headings, code and the items of a single-spaced list do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 19.8.

4.43 parenthesis pairs/response · raw 4.43 parenthesis pairs/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Parentheses: Closed pairs of parentheses, nested pairs included; unmatched parentheses do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 4.43.

5.02 question marks/response · raw 5.02 question marks/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Questions: Question marks. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 5.02.

12.50 quotation marks/response · raw 12.5 quotation marks/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Quotes: Quotation marks of any style, such as straight, curly, « » and „ “; apostrophes do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 12.5.

51.70 sentences/response · raw 51.7 sentences/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Sentences: Sentences found by a rule-based splitter that handles abbreviations such as “Dr.”. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 51.7.

0.09 tables/response · raw 0.09 tables/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Tables: Tables. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.09.

702 words/response · raw 702 words/response

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Words: Tokens that contain a letter or digit; each Chinese, Japanese or Korean character counts as one word (but all questions are in English). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 702.

83 percentile · raw 83 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Direct, confident positions and constructive pushback. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 83.

85 percentile · raw 85 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Rich, varied, well-crafted language. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 85.

88 percentile · raw 88 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Creative, exploratory, open-ended thinking. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 88.

94 percentile · raw 94 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Practical depth, evidence, actionable content. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 94.

85 percentile · raw 85 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Well-structured, scannable, answer-first. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 85.

51 percentile · raw 51 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Use of filler and generic language. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 51.

62 percentile · raw 62 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Truthful, acknowledges uncertainty, non-sycophantic. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 62.

100 percentile · raw 100 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Friendly, humorous, sincere, emotionally attuned. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 100.

100 percentile · raw 100 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Actionable Guidance writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 100.

49 percentile · raw 49 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Admits Uncertainty writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 49.

58 percentile · raw 58 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Answer First writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 58.

83 percentile · raw 83 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Bottom-Line Summary writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 83.

89 percentile · raw 89 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Confident writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 89.

83 percentile · raw 83 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Constructive Pushback writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 83.

92 percentile · raw 92 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Creative writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 92.

26 percentile · raw 26 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Direct writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 26.

52 percentile · raw 52 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Disclaims Assumptions writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 52.

86 percentile · raw 86 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Eloquent writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 86.

100 percentile · raw 100 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Emotionally Intelligent writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 100.

89 percentile · raw 89 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Evidence-Based writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 89.

86 percentile · raw 86 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Flags Nuances writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 86.

94 percentile · raw 94 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Friendly writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 94.

92 percentile · raw 92 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Humorous writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 92.

86 percentile · raw 86 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Information Dense writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 86.

86 percentile · raw 86 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Lexically Varied writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 86.

43 percentile · raw 43 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Literate writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 43.

77 percentile · raw 77 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Metaphorical writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 77.

89 percentile · raw 89 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Natural Sounding writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 89.

72 percentile · raw 72 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Open-Ended writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 72.

92 percentile · raw 92 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Opinionated writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 92.

100 percentile · raw 100 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Pragmatic writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 100.

86 percentile · raw 86 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Resource-Specific writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 86.

46 percentile · raw 46 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Scannable writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 46.

72 percentile · raw 72 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Sincere writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 72.

52 percentile · raw 52 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Slop writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 52.

77 percentile · raw 77 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Structured writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 77.

63 percentile · raw 63 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Sycophantic writing trait, judged pairwise among responses to the same questions. Native trait direction: higher means more sycophancy; only its contribution to Transparency is inverted. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 63.

75 percentile · raw 75 percentile

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Truthful writing trait, judged pairwise among responses to the same questions. A judged writing/personality trait in this question population, not a general factual-accuracy benchmark. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 75.

1770 · raw 1770 Elo

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Manually observed public GDPval-AA v2.1 leaderboard table, full 298-configuration pool checked 20 October 2026. Exact source configuration: Muse Wonder (Max); creator: Meta; source rank 5; source release-month label Oct 2026 is model release metadata, not execution date. Published rounded native Elo 1770; literal displayed CI -23 / +23. Native Elo is not a percentage and is not the normalized (Elo−500)/2000 chart quantity. This versioned metric is separate from older generic GDPval-AA Elo and vendor GDPval/GDPval-AA results; no old evidence is overwritten. Execution date, inference precision, unprinted effort budgets and task costs are unpublished in this captured table. Elo is relative to the dated comparison pool; drift from an earlier observation does not prove a new evaluation run. Headline: highest published native Elo for this exact approved model; tied values prefer existing featured effort then source order.

52.4 · raw 52.393894697293 index

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Native index/Elo scale.

1770.3 · raw 1770.34 Elo

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Native index/Elo scale.

25.5 · raw 25.483333333333 index

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Native index/Elo scale.

41.1% · raw 41.1 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Raw 0–1 score ×100.

26.5% · raw 26.513865308432 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Raw 0–1 score ×100.

52% · raw 52.020202020202 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Raw 0–1 score ×100.

60.3% · raw 60.300925925926 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Raw 0–1 score ×100. Evaluation marked under review on source.

80.7% · raw 80.666666666667 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Raw 0–1 score ×100.

50.1% · raw 50.092678405931 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Raw 0–1 score ×100.

30.6% · raw 30.6 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Raw 0–1 score ×100.

28.3% · raw 28.285714285714 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Raw 0–1 score ×100. Evaluation marked under review on source.

68.1% · raw 68.094551312078 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Raw 0–1 score ×100. Partial reward, not strict all-assertion completion (Meta quotes 39.1/45.4 strict).

51.3 · raw 51.32355432236 index

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Native capability index; not percent.

54.8 · raw 54.815537886439 index

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Native capability index; not percent.

52 · raw 51.958998050727 index

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Native capability index; not percent.

53.8 · raw 53.772239206777 index

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Native capability index; not percent.

57.7 · raw 57.694458275409 index

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Native capability index; not percent.

1658.7 · raw 1658.65 Elo

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. AA-Briefcase v1.1; rubric fraction converted to percent, quality/presentation retain Elo.

61.2% · raw 61.212121212121 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. AA-Briefcase v1.1; rubric fraction converted to percent, quality/presentation retain Elo.

1962.4 · raw 1962.43 Elo

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. AA-Briefcase v1.1; rubric fraction converted to percent, quality/presentation retain Elo.

1475.8 · raw 1475.81 Elo

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. AA-Briefcase v1.1; rubric fraction converted to percent, quality/presentation retain Elo.

$4.516/task · raw 4.515677444063 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Observed weighted cost per index task; not subscription charge.

$12.451/task · raw 12.450586286484 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

$4.888/task · raw 4.888460179864 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

$0.83/task · raw 0.829714331948 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

$17.968/task · raw 17.96813655 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

$0.135/task · raw 0.134993125 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

$0.572/task · raw 0.571862409639 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

$0.723/task · raw 0.72263932 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

$1.486/task · raw 1.486242971429 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

$0.05/task · raw 0.050124392667 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

$0.437/task · raw 0.4370334 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

AA exact public model-page data, max effort; checked 12 October 2026, run date unpublished. Intelligence Index v4.3.2; non-estimated. Source per-suite observed task cost, not weighted index contribution; harness-specific.

61.060% · raw 61.06 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±1.07 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

62.6% · raw 62.55 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±4.35 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

80.4% · raw 80.36 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±5.13 percentage points. Vals CyberBench v1.1 capability, not Meta refusal CyberBench. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

67.1% · raw 67.08 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±2.89 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

56.6% · raw 56.64 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±0.38 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

50.5% · raw 50.48 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±3.48 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

27.5% · raw 27.48 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±3.00 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

84% · raw 84 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±3.69 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

70.2% · raw 70.23 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±1.19 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

36.3% · raw 36.27 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±3.47 percentage points. Reviewed 2026-10-14: this legacy generic key mixes quantity labels across models. Current source explicitly identifies this historical value as Accuracy (published current raw 36.269). Keep stored metric/value unchanged for historical evidence; use newly separate citation-adjusted Accuracy and Weighted Pass metrics for like-for-like comparisons. No inferred model regression. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

67% · raw 67.04 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±0.37 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

17.1% · raw 17.08 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±3.10 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

66.9% · raw 66.94 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±4.91 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

49.5% · raw 49.49 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±2.20 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

22.9% · raw 22.86 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals exact Meta Wonder identity, model page linked by 12 October v2 Finance/Index boards; max effort. Default Meta provider, temperature 1, top_p/top_k default, 131,072 output-token harness ceiling; suite overrides possible. Run date unpublished. Published uncertainty ±5.05 percentage points. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

$11.43/task · raw 11.43 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Vals observed index cost/test; max effort. Model page full precision; dated evaluator valuation, not public API availability or a subscription bill. Run date unpublished. Citation repaired 2026-10-14: original https://www.vals.ai/models/meta_restful_rf120_max now returns 404; the exact renamed model page https://www.vals.ai/models/meta_muse-wonder reproduces this recorded value at its published precision. Historical score, effort and as_of are retained.

$2.67/task · raw 2.67 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-12

Published evaluator Cost / Test; max default, suite-specific harness. Vibe Code uses OpenHands. Not token prices or subscription charges; Full priced evaluator run does not establish a public API.

31.7 fixes · raw 31.7 fixes

Headline · max effort · Independent evaluator

Source/record date: 2026-10-13

Harness: Muse Code / Meta API; max effort; mean of 3 runs over two repositories, 105 planted bugs. Source row dated 2026-10-13; checked 13 October 2026. Only verified planted fixes count; partial, claimed-only and unplanted extras stay outside the score. Runs scored 30, 36 and 29. These overlap Muse Spark 1.3's reported range; the means alone do not establish a regression. Muse Code 1.4.1; max is Meta's published top tier, not magnitude-probed. The reminder observer uses fixed efforts. Six counted repo legs completed; four account-error legs were excluded, not assigned zero. API-equivalent list cost is not a subscription bill; some wall measurements include contention. Blind Codex gpt-5.5 judge.

$8932.18 · raw 8932.18 USD

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-14

Harness: Andon Vending-Bench 2; exact public configuration Muse Wonder. Source runs.vb 2.final_value is the publisher's arithmetic mean final bank balance over 6 year-long simulated runs, not a new average created by Examenos. Net money gain = published mean final balance $9432.18 minus the benchmark's $500 starting balance; $8932.18 net gain, rounded to cents. Mean final-balance std 2366.066333693964; SEM 965.9425358546611; 95% CI half-width 1893.2473702751356. Preserve standard deviation and CI semantics. This is simulated business money, not API cost, real revenue or subscription value. Source model effort and execution dates are not published in this data module. Vending-Bench 1 data are excluded.

10% · raw 10 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-14

Harness: Artificial Analysis Stirrup, Harvey private LAB v 1.1 120 tasks /24 practice areas, pass@1, up to 200 turns. Published exact configuration Muse Wonder (Max). Three rubric judges GPT-6 Sol/Grok 4.7/Claude Opus 5.5 medium; material hallucination checks GPT-6 Sol high. Material flags zero a task, minor flags do not. Current visible source table has 27 configurations, checked 14 Oct 2026; run date unpublished. This is independent Harvey v 1.1, not legacy HLAB(AA), Vals HLAB or Meta vendor-cited Harvey. Curated existing-metric refresh for current owner request; no future AA automated adapter authorization.

95.1% · raw 95.1 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-14

Harness: Artificial Analysis Stirrup, Harvey private LAB v 1.1 120 tasks /24 practice areas, pass@1, up to 200 turns. Published exact configuration Muse Wonder (Max). Three rubric judges GPT-6 Sol/Grok 4.7/Claude Opus 5.5 medium; material hallucination checks GPT-6 Sol high. Material flags zero a task, minor flags do not. Current visible source table has 27 configurations, checked 14 Oct 2026; run date unpublished. This is independent Harvey v 1.1, not legacy HLAB(AA), Vals HLAB or Meta vendor-cited Harvey. Curated existing-metric refresh for current owner request; no future AA automated adapter authorization.

0.89 hallucinations/task · raw 0.89 hallucinations/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-14

Harness: Artificial Analysis Stirrup, Harvey private LAB v 1.1 120 tasks /24 practice areas, pass@1, up to 200 turns. Published exact configuration Muse Wonder (Max). Three rubric judges GPT-6 Sol/Grok 4.7/Claude Opus 5.5 medium; material hallucination checks GPT-6 Sol high. Material flags zero a task, minor flags do not. Current visible source table has 27 configurations, checked 14 Oct 2026; run date unpublished. This is independent Harvey v 1.1, not legacy HLAB(AA), Vals HLAB or Meta vendor-cited Harvey. Curated existing-metric refresh for current owner request; no future AA automated adapter authorization. Mean material flags per checked usable submission, native count; not percent.

$13.86/task · raw 13.86 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-14

Harness: Artificial Analysis Stirrup, Harvey private LAB v 1.1 120 tasks /24 practice areas, pass@1, up to 200 turns. Published exact configuration Muse Wonder (Max). Three rubric judges GPT-6 Sol/Grok 4.7/Claude Opus 5.5 medium; material hallucination checks GPT-6 Sol high. Material flags zero a task, minor flags do not. Current visible source table has 27 configurations, checked 14 Oct 2026; run date unpublished. This is independent Harvey v 1.1, not legacy HLAB(AA), Vals HLAB or Meta vendor-cited Harvey. Curated existing-metric refresh for current owner request; no future AA automated adapter authorization. Published mean modeled API-equivalent task cost, not a bill or subscription capacity.

36.269% · raw 36.269 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-14

Harness: Vals Tax Agent Bench v 1, private 193-question corporate-tax test set; all questions equally weighted,3-89 rubric checks/question with 1-3 importance weights. Current source 14 Oct 2026, primary updated 12 Oct. Only final answer/citations graded; timed-out unanswered question zero. Citation factor=0.7+0.3*fraction of cited sources resolving to real authoritative documents. Accuracy requires every rubric check passing; any failed check zeroes task. Weighted Pass is weighted rubric points earned times citation factor, with any must-pass failure zeroing task (approximately 40% of checks must-pass). LLM-judged, separate from real professional correctness. This exact metric is kept separate from generic legacy Tax Agent Bench rows which mix quantity labels. Default model effortmax; individual suite overrides possible, run date unpublished. Model card displays ±3.47 percentage points; confidence level not inferred.

71.161% · raw 71.161 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-14

Harness: Vals Tax Agent Bench v 1, private 193-question corporate-tax test set; all questions equally weighted,3-89 rubric checks/question with 1-3 importance weights. Current source 14 Oct 2026, primary updated 12 Oct. Only final answer/citations graded; timed-out unanswered question zero. Citation factor=0.7+0.3*fraction of cited sources resolving to real authoritative documents. Accuracy requires every rubric check passing; any failed check zeroes task. Weighted Pass is weighted rubric points earned times citation factor, with any must-pass failure zeroing task (approximately 40% of checks must-pass). LLM-judged, separate from real professional correctness. This exact metric is kept separate from generic legacy Tax Agent Bench rows which mix quantity labels. Default model effortmax; individual suite overrides possible, run date unpublished.

39.1% · raw 39.12 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-17

Vals Science Index v1, published 16 October 2026. Equal weighting of BioMysteryBench, MysteryMechanism and Terminal-Bench-Science; aggregate copied directly, not computed from rounded components. Exact model label Wonder. Index-level effort/provider/run date not stated; do not infer from product defaults or other suites. Source auxiliary columns retained verbatim: cost/test $8.74, input/output token rates $4 / $20, input/output tokens 5.90B / 45.38M, duration 43m08s. These are evaluation context, not our public API pricing or subscription coefficients.

43.7% · raw 43.7 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-17

Scale Labs / Elorian primary live HSS board, frozen 522-task suite (288 images, 234 silent videos). Main pass@1 averages three attempts and requires all rubric criteria. Claude Opus 5 judge; task-cluster bootstrap 95% half-width ±4.2 percentage points. Board explicit effort max. Exact inference endpoint/output ceiling/frame configuration for these newly listed models is not published; blanket high-effort prose does not override the row. Entry createdAt 2026-10-14T17:02:13.000Z is source metadata, not proof of evaluation/publication date. Public-media contamination and judge dependence apply. Unversioned Qwen3.8-Max remains unmatched.