Examenos

62.2% · raw 62.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: 600 publicly released AutomationBench tasks; benchmark version 1.0.6, main-table harness and effort unspecified. Not the Claude Code effort sweep. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

62% · raw 62 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: launch-tweet rounded summary; not a distinct evaluation. The detailed launch table remains headline. Effort/configuration is not restated in this summary.

76.2% · raw 76.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: OpenCode 1.17.18; MiniMax-M2.7 judge. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

20.8% · raw 20.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Vendor explicitly attributes this figure to Artificial Analysis. It remains a vendor-cited observation; the exact new model was not found in the checked AA catalog. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

38.1% · raw 38.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Codex 0.149.1. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

60.3% · raw 60.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Official LOCA implementation at 256K. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

29.6% · raw 29.6 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Official ALE Overall full 152-task population, not ALE-CLI or an unlicensed subset. Vendor quotes the public board; direct board results are recorded independently. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

29.6% · raw 29.6 %

Alternative · max effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code, max. ALE-v1 Overall 152-task full population. Explicit vendor label; independent public source confirms rounded pass rate. Chart cost estimate is a different metric; board costs are withheld. Source public-model-data date 18 October, evaluation date unknown.

30% · raw 30 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: launch-tweet rounded summary; not a distinct evaluation. The detailed launch table remains headline. Effort/configuration is not restated in this summary.

61.7% · raw 61.7 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Official 0808 full release; source says highest release-time report for peers except the DeepSeek V4.1 in-house reproduction. Reward type and exact action budget unspecified. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

39.3 score · raw 39.3 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code 2.1.251; maximum context 512K; PostTrainBench v1.1, not v1.2. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

73.7% · raw 73.7 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: mini-swe-agent 2.4.5; maximum context 512K. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

73.7% · raw 73.7 %

Alternative · max effort · Own vendor

Source/record date: 2026-10-19

Harness: mini-swe-agent, explicit labeled effort sweep. Main table states version 2.4.5 and 512K context; chart does not restate those settings. Retain table headline separately. The footnote says public data as of 22 September despite some later-named peers; this does not establish run dates. Only explicit numeric labels are imported, never unlabeled peer scores or dollar coordinates.

72.1% · raw 72.1 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-10-19

Harness: mini-swe-agent, explicit labeled effort sweep. Main table states version 2.4.5 and 512K context; chart does not restate those settings. Retain table headline separately. The footnote says public data as of 22 September despite some later-named peers; this does not establish run dates. Only explicit numeric labels are imported, never unlabeled peer scores or dollar coordinates.

68.1% · raw 68.1 %

Alternative · high effort · Own vendor

Source/record date: 2026-10-19

Harness: mini-swe-agent, explicit labeled effort sweep. Main table states version 2.4.5 and 512K context; chart does not restate those settings. Retain table headline separately. The footnote says public data as of 22 September despite some later-named peers; this does not establish run dates. Only explicit numeric labels are imported, never unlabeled peer scores or dollar coordinates.

74% · raw 74 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: launch-tweet rounded summary; not a distinct evaluation. The detailed launch table remains headline. Effort/configuration is not restated in this summary.

61.2% · raw 61.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code 2.1.233; maximum context 512K; reward hacking assigned zero; two task specification/test inconsistencies corrected. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

23% · raw 23 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code 2.1.220; maximum context 200K. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

82.2% · raw 82.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code 2.1.263; maximum context 1M. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

82% · raw 82 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: same launch claim, rounded prose value 82%; the table reports 82.2 with more precision and remains headline. This is not a second independently executed run.

91% · raw 91 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code 2.1.285; minimum image side 1,500 pixels; tools enabled. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

91% · raw 91 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: launch-tweet rounded summary; not a distinct evaluation. The detailed launch table remains headline. Effort/configuration is not restated in this summary.

43.4% · raw 43.4 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: 2,158 text-only HLE questions, no tools. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

82.5% · raw 82.5 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: GPT-5.5 judge. Own vendor launch figure. Main figure contains no per-model effort labels. Execution date, exact checkpoint SHA, generation budget and sampling settings are unpublished. Source checked 2026-10-19.

62.8% · raw 62.8 %

Headline · max effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code, AutomationBench v1.0.6. Explicit labeled effort sweep; max is the headline. Source footnote public-model-data date 18 October is not an execution date. Main table 62.2 is a different unspecified setup, not silently replaced. No cost coordinate is estimated.

61.2% · raw 61.2 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code, AutomationBench v1.0.6. Explicit labeled effort sweep; max is the headline. Source footnote public-model-data date 18 October is not an execution date. Main table 62.2 is a different unspecified setup, not silently replaced. No cost coordinate is estimated.

59.2% · raw 59.2 %

Alternative · high effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code, AutomationBench v1.0.6. Explicit labeled effort sweep; max is the headline. Source footnote public-model-data date 18 October is not an execution date. Main table 62.2 is a different unspecified setup, not silently replaced. No cost coordinate is estimated.

57.7% · raw 57.7 %

Alternative · medium effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code, AutomationBench v1.0.6. Explicit labeled effort sweep; max is the headline. Source footnote public-model-data date 18 October is not an execution date. Main table 62.2 is a different unspecified setup, not silently replaced. No cost coordinate is estimated.

52.8% · raw 52.8 %

Alternative · low effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code, AutomationBench v1.0.6. Explicit labeled effort sweep; max is the headline. Source footnote public-model-data date 18 October is not an execution date. Main table 62.2 is a different unspecified setup, not silently replaced. No cost coordinate is estimated.

$111.00 · raw 111 USD

Headline · max effort · Own vendor

Source/record date: 2026-10-19

Harness: Claude Code max, ALE v1 Overall 152-task suite. Explicit $111 vendor-estimated total, not an invoice or per-task figure. Independent ALE frontend deliberately hides MiniMax costs; API database numbers are not published as independent costs.

53.781% · raw 53.781 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2597.835,"stderr":1.067,"cost_per_test":3.866285,"token_totals":{"input_tokens":18180722777,"output_tokens":306409526,"reasoning_tokens":243120444,"cache_read_tokens":17662797762,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

$3.866285/task · raw 3.866285 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2597.835,"stderr":1.067,"cost_per_test":3.866285,"token_totals":{"input_tokens":18180722777,"output_tokens":306409526,"reasoning_tokens":243120444,"cache_read_tokens":17662797762,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

2597.835 s · raw 2597.835 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2597.835,"stderr":1.067,"cost_per_test":3.866285,"token_totals":{"input_tokens":18180722777,"output_tokens":306409526,"reasoning_tokens":243120444,"cache_read_tokens":17662797762,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

73.944% · raw 73.944 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1263.916,"stderr":4.249,"cost_per_test":8.568544,"token_totals":{"input_tokens":3334725307,"output_tokens":8488142,"reasoning_tokens":2462335,"cache_read_tokens":3318290676,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":"OpenHands"}

58.826% · raw 58.826 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":904.204,"stderr":2.07,"cost_per_test":0.158978,"token_totals":{"input_tokens":469331836,"output_tokens":54339785,"reasoning_tokens":45403914,"cache_read_tokens":234241552,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

67.843% · raw 67.843 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

48.195% · raw 48.195 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":7363.872,"stderr":3.35,"cost_per_test":12.723188,"token_totals":{"input_tokens":6238818064,"output_tokens":20288643,"reasoning_tokens":12068935,"cache_read_tokens":6202642760,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

34.135% · raw 34.135 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2911.983,"stderr":3.296,"cost_per_test":2.106744,"token_totals":{"input_tokens":1230685744,"output_tokens":102073269,"reasoning_tokens":90583933,"cache_read_tokens":1135801046,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

7.500% · raw 7.5 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1497.601,"stderr":2.265,"cost_per_test":0.472848,"token_totals":{"input_tokens":2730849267,"output_tokens":20117381,"reasoning_tokens":6681765,"cache_read_tokens":2703378118,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

25.253% · raw 25.253 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3030.551,"stderr":1.01,"cost_per_test":5.202085,"token_totals":{"input_tokens":2497428556,"output_tokens":16603505,"reasoning_tokens":13158836,"cache_read_tokens":2489516469,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

32.136% · raw 32.136 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1962.771,"stderr":3.342,"cost_per_test":1.428933,"token_totals":{"input_tokens":551883325,"output_tokens":70312257,"reasoning_tokens":62154212,"cache_read_tokens":466865454,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

58.826% · raw 58.826 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":904.204,"stderr":2.07,"cost_per_test":0.158978,"token_totals":{"input_tokens":469331836,"output_tokens":54339785,"reasoning_tokens":45403914,"cache_read_tokens":234241552,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

$0.158978/task · raw 0.158978 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":904.204,"stderr":2.07,"cost_per_test":0.158978,"token_totals":{"input_tokens":469331836,"output_tokens":54339785,"reasoning_tokens":45403914,"cache_read_tokens":234241552,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

904.204 s · raw 904.204 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":904.204,"stderr":2.07,"cost_per_test":0.158978,"token_totals":{"input_tokens":469331836,"output_tokens":54339785,"reasoning_tokens":45403914,"cache_read_tokens":234241552,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

45.595% · raw 45.595 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":904.204,"stderr":2.206,"cost_per_test":0.158978,"token_totals":{"input_tokens":469331836,"output_tokens":54339785,"reasoning_tokens":45403914,"cache_read_tokens":234241552,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

73.855% · raw 73.855 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1265.243,"stderr":5.425,"cost_per_test":0.223672,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

73.741% · raw 73.741 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":722.092,"stderr":5.482,"cost_per_test":0.142236,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

55.738% · raw 55.738 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":795.075,"stderr":6.969,"cost_per_test":0.125556,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

53.008% · raw 53.008 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1155.838,"stderr":6.676,"cost_per_test":0.2003,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

33.919% · raw 33.919 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1225.044,"stderr":6.7,"cost_per_test":0.222412,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

59.658% · raw 59.658 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":812.371,"stderr":6.963,"cost_per_test":0.144584,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

82.124% · raw 82.124 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":643.947,"stderr":5.244,"cost_per_test":0.078555,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

70.509% · raw 70.509 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1102.958,"stderr":6.171,"cost_per_test":0.19942,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

26.881% · raw 26.881 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":513.254,"stderr":5.969,"cost_per_test":0.104852,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

67.843% · raw 67.843 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

$0.268958/task · raw 0.268958 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

1847.780 s · raw 1847.78 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

71.645% · raw 71.645 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1872.953,"stderr":2.773,"cost_per_test":0.295804,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

63.888% · raw 63.888 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1822.607,"stderr":2.408,"cost_per_test":0.242112,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

58.626% · raw 58.626 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3945.157,"stderr":4.241,"cost_per_test":0.546001,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

58.739% · raw 58.739 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2759.397,"stderr":6.7,"cost_per_test":0.244351,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

74.391% · raw 74.391 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2537.751,"stderr":3.297,"cost_per_test":0.195966,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

66.456% · raw 66.456 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":861.196,"stderr":4.786,"cost_per_test":0.189071,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

71.581% · raw 71.581 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":770.52,"stderr":3.537,"cost_per_test":0.255073,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

80.223% · raw 80.223 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":826.821,"stderr":5.877,"cost_per_test":0.175491,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

64.885% · raw 64.885 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1125.671,"stderr":2.834,"cost_per_test":0.247506,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

64.846% · raw 64.846 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

52.407% · raw 52.407 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

60.973% · raw 60.973 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

56.505% · raw 56.505 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

76.446% · raw 76.446 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

72.337% · raw 72.337 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

72.048% · raw 72.048 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

60.864% · raw 60.864 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

75.467% · raw 75.467 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

67.695% · raw 67.695 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

84.148% · raw 84.148 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

76.298% · raw 76.298 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

67.341% · raw 67.341 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

62.430% · raw 62.43 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

55.232% · raw 55.232 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

84.866% · raw 84.866 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

73.299% · raw 73.299 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1847.78,"stderr":2.29,"cost_per_test":0.268958,"token_totals":{"input_tokens":1127000678,"output_tokens":14186544,"reasoning_tokens":10606514,"cache_read_tokens":1112061687,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

7.500% · raw 7.5 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1497.601,"stderr":2.265,"cost_per_test":0.472848,"token_totals":{"input_tokens":2730849267,"output_tokens":20117381,"reasoning_tokens":6681765,"cache_read_tokens":2703378118,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

$0.472848/task · raw 0.472848 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1497.601,"stderr":2.265,"cost_per_test":0.472848,"token_totals":{"input_tokens":2730849267,"output_tokens":20117381,"reasoning_tokens":6681765,"cache_read_tokens":2703378118,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

1497.601 s · raw 1497.601 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1497.601,"stderr":2.265,"cost_per_test":0.472848,"token_totals":{"input_tokens":2730849267,"output_tokens":20117381,"reasoning_tokens":6681765,"cache_read_tokens":2703378118,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1602.082,"stderr":0.0,"cost_per_test":0.447689,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1490.442,"stderr":0.0,"cost_per_test":0.572096,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1557.112,"stderr":0.0,"cost_per_test":0.363031,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1586.546,"stderr":0.0,"cost_per_test":0.501587,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1222.418,"stderr":0.0,"cost_per_test":0.336279,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2097.538,"stderr":20.0,"cost_per_test":0.628725,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1048.75,"stderr":0.0,"cost_per_test":0.385034,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1603.698,"stderr":12.247,"cost_per_test":0.652971,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1523.606,"stderr":0.0,"cost_per_test":0.437811,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1855.334,"stderr":0.0,"cost_per_test":0.426654,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

40.000% · raw 40 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1434.758,"stderr":24.495,"cost_per_test":0.616064,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1706.268,"stderr":0.0,"cost_per_test":0.440838,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

10.000% · raw 10 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1901.926,"stderr":10.0,"cost_per_test":0.565023,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1851.156,"stderr":20.0,"cost_per_test":0.526994,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1131.4,"stderr":20.0,"cost_per_test":0.37483,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1028.702,"stderr":0.0,"cost_per_test":0.343356,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1452.402,"stderr":20.0,"cost_per_test":0.496673,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1530.654,"stderr":0.0,"cost_per_test":0.355021,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

10.000% · raw 10 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1470.038,"stderr":10.0,"cost_per_test":0.48408,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1428.066,"stderr":0.0,"cost_per_test":0.471568,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

20.000% · raw 20 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1092.26,"stderr":20.0,"cost_per_test":0.430794,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1483.114,"stderr":0.0,"cost_per_test":0.548613,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1486.298,"stderr":0.0,"cost_per_test":0.4795,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

0.000% · raw 0 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1357.85,"stderr":0.0,"cost_per_test":0.463122,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

91.077% · raw 91.077 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1497.601,"stderr":0.343,"cost_per_test":0.472848,"token_totals":{"input_tokens":2730849267,"output_tokens":20117381,"reasoning_tokens":6681765,"cache_read_tokens":2703378118,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

92.029% · raw 92.029 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1602.082,"stderr":1.633,"cost_per_test":0.447689,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

88.351% · raw 88.351 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1490.442,"stderr":1.924,"cost_per_test":0.572096,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

93.657% · raw 93.657 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1557.112,"stderr":1.492,"cost_per_test":0.363031,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

87.716% · raw 87.716 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1586.546,"stderr":1.934,"cost_per_test":0.501587,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

92.353% · raw 92.353 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1222.418,"stderr":1.667,"cost_per_test":0.336279,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

91.438% · raw 91.438 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2097.538,"stderr":1.64,"cost_per_test":0.628725,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

96.302% · raw 96.302 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1048.75,"stderr":1.072,"cost_per_test":0.385034,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

96.820% · raw 96.82 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1603.698,"stderr":1.045,"cost_per_test":0.652971,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

94.539% · raw 94.539 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1523.606,"stderr":1.33,"cost_per_test":0.437811,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

82.210% · raw 82.21 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1855.334,"stderr":2.345,"cost_per_test":0.426654,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

95.875% · raw 95.875 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1434.758,"stderr":1.144,"cost_per_test":0.616064,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

89.427% · raw 89.427 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1706.268,"stderr":1.844,"cost_per_test":0.440838,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

87.901% · raw 87.901 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1901.926,"stderr":1.763,"cost_per_test":0.565023,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

95.134% · raw 95.134 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1851.156,"stderr":1.248,"cost_per_test":0.526994,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

90.727% · raw 90.727 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1131.4,"stderr":1.752,"cost_per_test":0.37483,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

90.074% · raw 90.074 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1028.702,"stderr":1.816,"cost_per_test":0.343356,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

92.349% · raw 92.349 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1452.402,"stderr":1.589,"cost_per_test":0.496673,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

91.434% · raw 91.434 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1530.654,"stderr":1.77,"cost_per_test":0.355021,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

91.917% · raw 91.917 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1470.038,"stderr":1.674,"cost_per_test":0.48408,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

92.420% · raw 92.42 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1428.066,"stderr":1.431,"cost_per_test":0.471568,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

87.581% · raw 87.581 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1092.26,"stderr":1.876,"cost_per_test":0.430794,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

84.281% · raw 84.281 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1483.114,"stderr":2.108,"cost_per_test":0.548613,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

89.504% · raw 89.504 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1486.298,"stderr":1.897,"cost_per_test":0.4795,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

91.414% · raw 91.414 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Harvey held-out legal work protocol, internet disabled, GPT5.5 medium and Sonnet4.6 judges. Task pass requires every criterion; final is mean of two judges. Criteria-pass rate is separate. DOCX tracked-change grading fix. Distinct from AA hallucination-gated LAB. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1357.85,"stderr":1.628,"cost_per_test":0.463122,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

32.136% · raw 32.136 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1962.771,"stderr":3.342,"cost_per_test":1.428933,"token_totals":{"input_tokens":551883325,"output_tokens":70312257,"reasoning_tokens":62154212,"cache_read_tokens":466865454,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

$1.428933/task · raw 1.428933 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1962.771,"stderr":3.342,"cost_per_test":1.428933,"token_totals":{"input_tokens":551883325,"output_tokens":70312257,"reasoning_tokens":62154212,"cache_read_tokens":466865454,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

1962.771 s · raw 1962.771 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1962.771,"stderr":3.342,"cost_per_test":1.428933,"token_totals":{"input_tokens":551883325,"output_tokens":70312257,"reasoning_tokens":62154212,"cache_read_tokens":466865454,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

31.860% · raw 31.86 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1831.299,"stderr":7.108,"cost_per_test":1.25238,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

36.667% · raw 36.667 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2279.728,"stderr":8.949,"cost_per_test":1.823243,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

31.202% · raw 31.202 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2472.42,"stderr":8.292,"cost_per_test":1.70664,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

59.875% · raw 59.875 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1323.436,"stderr":9.079,"cost_per_test":0.869841,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

26.667% · raw 26.667 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1834.474,"stderr":8.212,"cost_per_test":1.399246,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

6.667% · raw 6.667 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2129.255,"stderr":4.632,"cost_per_test":1.598752,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

62.715% · raw 62.715 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1962.771,"stderr":3.255,"cost_per_test":1.428933,"token_totals":{"input_tokens":551883325,"output_tokens":70312257,"reasoning_tokens":62154212,"cache_read_tokens":466865454,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

50.785% · raw 50.785 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1831.299,"stderr":7.357,"cost_per_test":1.25238,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

57.664% · raw 57.664 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2279.728,"stderr":8.78,"cost_per_test":1.823243,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

58.878% · raw 58.878 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2472.42,"stderr":8.436,"cost_per_test":1.70664,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

84.510% · raw 84.51 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1323.436,"stderr":6.192,"cost_per_test":0.869841,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

75.287% · raw 75.287 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1834.474,"stderr":7.105,"cost_per_test":1.399246,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

54.339% · raw 54.339 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2129.255,"stderr":8.313,"cost_per_test":1.598752,"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

73.944% · raw 73.944 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. Independent model-plus-harness web-application evaluation. Preserve OpenHands versus other agent variants; source costs/runtime are task observations, not API tariffs. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1263.916,"stderr":4.249,"cost_per_test":8.568544,"token_totals":{"input_tokens":3334725307,"output_tokens":8488142,"reasoning_tokens":2462335,"cache_read_tokens":3318290676,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":"OpenHands"}

$8.568544/task · raw 8.568544 USD/task

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. Independent model-plus-harness web-application evaluation. Preserve OpenHands versus other agent variants; source costs/runtime are task observations, not API tariffs. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1263.916,"stderr":4.249,"cost_per_test":8.568544,"token_totals":{"input_tokens":3334725307,"output_tokens":8488142,"reasoning_tokens":2462335,"cache_read_tokens":3318290676,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":"OpenHands"} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

1263.916 s · raw 1263.916 s

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. Independent model-plus-harness web-application evaluation. Preserve OpenHands versus other agent variants; source costs/runtime are task observations, not API tariffs. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1263.916,"stderr":4.249,"cost_per_test":8.568544,"token_totals":{"input_tokens":3334725307,"output_tokens":8488142,"reasoning_tokens":2462335,"cache_read_tokens":3318290676,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":"OpenHands"} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

2.59 acronyms/response · raw 2.59 acronyms/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Acronyms: Two or more capital letters in a row (AI, USA) or dotted forms (U.S.). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 2.59.

15.50 words/sentence · raw 15.5 words/sentence

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Avg. sentence length: Words divided by sentences. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 15.5.

10.60 bold spans/response · raw 10.6 bold spans/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Bold text: Uses of bold: each unbroken run of bold text counts once, so a single bolded word and a fully bolded sentence both count as 1. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 10.6.

3361 characters/response · raw 3361 characters/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Characters: Characters in the prose, spaces included, after code and URLs are removed. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 3,361.

7.63 colons/response · raw 7.63 colons/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Colons: Colons. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 7.63.

3.42 contractions/response · raw 3.42 contractions/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Contractions: Contractions such as “don’t”, “you’re” and “it’s”; possessives such as “John’s” do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 3.42.

0.06 ellipses/response · raw 0.06 ellipses/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Ellipses: Three periods (...) or the ellipsis character (…). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.06.

2.85 em dashes/response · raw 2.85 em dashes/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Em dashes: The em dash character (—); double hyphens do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 2.85.

0.01 emojis/response · raw 0.01 emojis/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Emojis: Each emoji as the reader sees it. An emoji built from several characters, such as a family (👨‍👩‍👧), still counts as one emoji. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.01.

2% · raw 2 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Ends with a question: Whether the answer’s last character is a question mark, reported as the share of responses that do. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 2%.

0.02 exclamations/response · raw 0.02 exclamations/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Exclamations: Exclamation marks. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.02.

3.51 headings/response · raw 3.51 headings/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Headings: Headings of any level. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 3.51.

0.56 italic spans/response · raw 0.56 italic spans/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Italics: Uses of italics: each unbroken run of italic text counts once, so a single italicized word and a fully italicized sentence both count as 1. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.56.

3.04 lists/response · raw 3.04 lists/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Lists: Top-level bulleted or numbered lists; nested sub-lists do not count separately. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 3.04.

11.10 paragraphs/response · raw 11.1 paragraphs/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Paragraphs: Paragraphs of prose; headings, code and the items of a single-spaced list do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 11.1.

0.47 parenthesis pairs/response · raw 0.47 parenthesis pairs/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Parentheses: Closed pairs of parentheses, nested pairs included; unmatched parentheses do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.47.

2.05 question marks/response · raw 2.05 question marks/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Questions: Question marks. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 2.05.

7.74 quotation marks/response · raw 7.74 quotation marks/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Quotes: Quotation marks of any style, such as straight, curly, « » and „ “; apostrophes do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 7.74.

33.10 sentences/response · raw 33.1 sentences/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Sentences: Sentences found by a rule-based splitter that handles abbreviations such as “Dr.”. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 33.1.

0.22 tables/response · raw 0.22 tables/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Tables: Tables. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.22.

484 words/response · raw 484 words/response

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Words: Tokens that contain a letter or digit; each Chinese, Japanese or Korean character counts as one word (but all questions are in English). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 484.

1474 · raw 1474 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Native combined analytical-quality, presentation and rubric Elo; separate from old unversioned AA-Briefcase records. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

47.8% · raw 47.8 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Task-level binary rubric pass rate; separate from analytical/presentation preference Elo. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

1652 · raw 1652 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Analytical quality preference Elo, separate from presentation and combined Elo. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

1425 · raw 1425 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Presentation preference Elo, separate from analytical quality and combined Elo. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

16 · raw 16 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Signed knowledge reliability index, not percent accuracy; correct answers rewarded and hallucinations penalized. Rounded public chart observation. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

56% · raw 56 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Share of all questions answered correctly in AA’s own knowledge evaluation. Rounded public chart label, not the vendor’s public-set test. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

93% · raw 93 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Incorrect / (incorrect + partial + not attempted), as defined by the source. Lower is better; not overall error fraction. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

73% · raw 73 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA agentic SaaS workflow objective reward. Separate from a vendor’s strict all-assertion task pass rate. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

48% · raw 48 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA’s published Humanity’s Last Exam observation; exact tools/population details are not restated on the captured model chart. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

21% · raw 21 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA professional-document reasoning all-pass rate. Distinct from GDPval work-product Elo. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

26% · raw 26 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA’s Terminal-Bench4.0 model-plus-agent setup, separate from vendor harness observations. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

80% · raw 80 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Version1.1 of AA long-context reasoning. Separate from old unversioned AA-LCR observations. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

53% · raw 53 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA scientist-authored coding evaluation. Source currently labels this benchmark under review; preserve that uncertainty. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

22% · raw 22 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA research-level physics evaluation. Source currently labels this benchmark under review; preserve that uncertainty. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

79% · raw 79 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA visual-reasoning observation for the exact named hosted model; serving precision, image pathway and execution date unpublished. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

43 · raw 43 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA Intelligence Index v4.3.2; public rounded summary. Exact measurement date and serving configuration remain unknown. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

120.1 tok/s · raw 120.1 tok/s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA observed output speed, separate from OpenRouter endpoint measurements or vendor throughput. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

$0.22 · raw 0.22 USD

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. AA usage-priced task cost, not an API token tariff, invoice or subscription coefficient. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred.

180 M tokens · raw 180 M tokens

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Source-rounded total output-token use for the AA Intelligence Index, in millions; not per-request output allowance. Manually checked rendered chart/summary, not an API import. Displayed rounding is preserved; no hidden precision, confidence interval, inference date or weight SHA is inferred. Source says reasoning enabled but does not publish an effort label. Promised weights remain unavailable; no license/size is inferred. Summary uses a source-rounded millions-of-output-tokens unit.

49 · raw 49 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Manual native chart observation, current source industry aggregate; no computed average or hidden precision inferred. Components: 7 evaluations: AA-Omniscience, GDPval-AA v2.1, AA-Briefcase v1.1, Humanity's Last Exam, AutomationBench-AA, AA-LCR v1.1, GDP.pdf. Displayed index is not percent task success; current snapshot remains separate from older unversioned industry observations.

55 · raw 55 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Manual native chart observation, current source industry aggregate; no computed average or hidden precision inferred. Components: 6 evaluations: GDPval-AA v2.1, AA-Briefcase v1.1, AA-Omniscience, AutomationBench-AA, AA-LCR v1.1, GDP.pdf. Displayed index is not percent task success; current snapshot remains separate from older unversioned industry observations.

47 · raw 47 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Manual native chart observation, current source industry aggregate; no computed average or hidden precision inferred. Components: 7 evaluations: AA-Omniscience, GDPval-AA v2.1, AA-Briefcase v1.1, Humanity's Last Exam, AA-LCR v1.1, GDP.pdf, AutomationBench-AA. Displayed index is not percent task success; current snapshot remains separate from older unversioned industry observations.

43 · raw 43 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Manual native chart observation, current source industry aggregate; no computed average or hidden precision inferred. Components: 6 evaluations: AA-Omniscience, Humanity's Last Exam, CritPt, GDPval-AA v2.1, AA-Briefcase v1.1, Terminal-Bench 4.0. Displayed index is not percent task success; current snapshot remains separate from older unversioned industry observations.

50 · raw 50 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Artificial Analysis published model evaluation. Manual native chart observation, current source industry aggregate; no computed average or hidden precision inferred. Components: 5 evaluations: AA-Omniscience, Humanity's Last Exam, GDPval-AA v2.1, AA-Briefcase v1.1, AA-LCR v1.1. Displayed index is not percent task success; current snapshot remains separate from older unversioned industry observations.

1458 · raw 1458 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Manually observed public GDPval-AA v2.1 leaderboard table, full 298-configuration pool checked 20 October 2026. Exact source configuration: MiniMax-M3.1-Flash; creator: MiniMax; source rank 59; source release-month label Oct 2026 is model release metadata, not execution date. Published rounded native Elo 1458; literal displayed CI -20 / +20. Native Elo is not a percentage and is not the normalized (Elo−500)/2000 chart quantity. This versioned metric is separate from older generic GDPval-AA Elo and vendor GDPval/GDPval-AA results; no old evidence is overwritten. Execution date, inference precision, unprinted effort budgets and task costs are unpublished in this captured table. Elo is relative to the dated comparison pool; drift from an earlier observation does not prove a new evaluation run. Source does not label an effort; unknown is preserved rather than copying a vendor default. Headline: highest published native Elo for this exact approved model; tied values prefer existing featured effort then source order.

31.2% · raw 31.20567375886525 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Overall · Unlicensed. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 44 passes; 140 tasks with submissions out of 141 split tasks; 140 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

28.4% · raw 28.368794326241137 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Overall · Unlicensed. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 40 passes; 137 tasks with submissions out of 141 split tasks; 137 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

57% · raw 56.99078106759686 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Overall · Unlicensed. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 44 passes; 140 tasks with submissions out of 141 split tasks; 140 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

54.3% · raw 54.33797644719029 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Overall · Unlicensed. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 40 passes; 137 tasks with submissions out of 141 split tasks; 137 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

46.3% · raw 46.26865671641791 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Near-term · Unlicensed. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 31 passes; 67 tasks with submissions out of 67 split tasks; 67 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

40.3% · raw 40.298507462686565 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Near-term · Unlicensed. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 27 passes; 67 tasks with submissions out of 67 split tasks; 67 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

77.2% · raw 77.2285154161045 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Near-term · Unlicensed. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 31 passes; 67 tasks with submissions out of 67 split tasks; 67 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

73.3% · raw 73.33131926155338 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Near-term · Unlicensed. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 27 passes; 67 tasks with submissions out of 67 split tasks; 67 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

6.7% · raw 6.666666666666667 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Last-Exam · Unlicensed. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 2 passes; 29 tasks with submissions out of 30 split tasks; 29 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

3.3% · raw 3.3333333333333335 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Last-Exam · Unlicensed. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 1 passes; 27 tasks with submissions out of 30 split tasks; 27 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

20.7% · raw 20.65754794594124 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Last-Exam · Unlicensed. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 2 passes; 29 tasks with submissions out of 30 split tasks; 29 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

16.3% · raw 16.261682814635485 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Last-Exam · Unlicensed. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 1 passes; 27 tasks with submissions out of 30 split tasks; 27 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

26% · raw 26 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Full-Spectrum · Unlicensed. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 13 passes; 48 tasks with submissions out of 50 split tasks; 48 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

24% · raw 24 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Full-Spectrum · Unlicensed. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 12 passes; 50 tasks with submissions out of 50 split tasks; 50 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

48.6% · raw 48.552116081813764 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Full-Spectrum · Unlicensed. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 13 passes; 48 tasks with submissions out of 50 split tasks; 48 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

49.5% · raw 49.4692479090425 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Full-Spectrum · Unlicensed. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 12 passes; 50 tasks with submissions out of 50 split tasks; 50 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

26.7% · raw 26.666666666666668 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 ALE-CLI · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 28 passes; 104 tasks with submissions out of 105 split tasks; 104 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

25.7% · raw 25.71428571428571 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 ALE-CLI · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 27 passes; 102 tasks with submissions out of 105 split tasks; 102 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

53.9% · raw 53.91510889246521 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 ALE-CLI · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 28 passes; 104 tasks with submissions out of 105 split tasks; 104 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

53% · raw 53.0244505351067 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 ALE-CLI · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 27 passes; 102 tasks with submissions out of 105 split tasks; 102 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

29.6% · raw 29.605263157894733 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Overall · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 45 passes; 149 tasks with submissions out of 152 split tasks; 149 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

27% · raw 26.973684210526315 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Overall · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 41 passes; 147 tasks with submissions out of 152 split tasks; 147 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

55% · raw 54.95140832888198 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Overall · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 45 passes; 149 tasks with submissions out of 152 split tasks; 149 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

52.9% · raw 52.927944585323175 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Overall · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 41 passes; 147 tasks with submissions out of 152 split tasks; 147 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

46.3% · raw 46.26865671641791 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Near-term · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 31 passes; 67 tasks with submissions out of 67 split tasks; 67 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

40.3% · raw 40.298507462686565 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Near-term · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 27 passes; 67 tasks with submissions out of 67 split tasks; 67 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

77.2% · raw 77.2285154161045 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Near-term · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 31 passes; 67 tasks with submissions out of 67 split tasks; 67 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

73.3% · raw 73.33131926155338 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Near-term · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 27 passes; 67 tasks with submissions out of 67 split tasks; 67 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

5.3% · raw 5.263157894736842 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Last-Exam · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 2 passes; 35 tasks with submissions out of 38 split tasks; 35 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

5.3% · raw 5.263157894736842 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Last-Exam · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 2 passes; 34 tasks with submissions out of 38 split tasks; 34 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

19.7% · raw 19.7010624693984 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Last-Exam · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 2 passes; 35 tasks with submissions out of 38 split tasks; 35 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

18% · raw 18.032720588272515 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Last-Exam · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 2 passes; 34 tasks with submissions out of 38 split tasks; 34 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

25.5% · raw 25.454545454545453 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Full-Spectrum · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 14 passes; 55 tasks with submissions out of 55 split tasks; 55 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

25.5% · raw 25.454545454545453 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Full-Spectrum · Full. Perfect-score pass rate, not partial credit. Raw fraction converted to percent. 14 passes; 53 tasks with submissions out of 55 split tasks; 53 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

50.2% · raw 50.20840719003864 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: claude_code; API variant reasoning-max; board effort label max. ALE-v1 Full-Spectrum · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 14 passes; 55 tasks with submissions out of 55 split tasks; 55 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.

49.3% · raw 49.33828734710342 %

Alternative · max effort · Independent evaluator

Source/record date: 2026-10-19

Harness: codex; API variant reasoning-max; board effort label max. ALE-v1 Full-Spectrum · Full. Published average partial-credit score, not perfect-score pass rate. Raw fraction converted to percent. 14 passes; 53 tasks with submissions out of 55 split tasks; 53 runs. Missing submissions do not prove a different task population. The publisher default configuration ranks pass rate, then partial score, coverage and effort; both headline metrics share it. Different agents/efforts remain separate alternatives. Runtime totals exclude queue/setup/grading/sync. No executed weight SHA or run date published; source check is 2026-10-19. MiniMax total costs are hidden by the public board; API DB amounts are not imported.