Examenos

39 · raw 39 index

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: AA Intelligence Index v4.3, cited by Meta as sourced 7 October 2026; direct AA model result unavailable at check time. Source date is not an execution date. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

61.8% · raw 61.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 65 tasks across 35 occupations; Meta cites official board; execution date unknown. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

89.2% · raw 89.2 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Scale evaluation pipeline: 1,000 tasks, 36 servers, 220 tools; pass when mean claim coverage >=0.75. Meta cites Scale; Beam self-reported. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

1401 · raw 1401 Elo

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: AA Stirrup/E2B harness, 220 tasks, 250 turns; vendor-cited Elo snapshot dated 7 October 2026, not a verified execution date. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

80.4% · raw 80.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: GUI-only, 361 tasks, 200 steps, partial reward mean over four attempts. Muse internal; Kimi self-report; do not assume identical run budgets for peer. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

72.8% · raw 72.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 108-task revised Toolathlon-Verified; Muse via official evaluation service; Kimi leaderboard and GLM self-report as cited by Meta. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

26.1% · raw 26.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 1,000 questions (500 reasoning/500 knowledge), closed book; sourced by Meta from Scale, not full HLE. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

25 · raw 25 index

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Signed Omniscience Index =100*(correct-incorrect)/all; not percent accuracy. Meta cites AA; Beam uses public split. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

58.5% · raw 58.5 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 288 test subproblems, all unit tests required; Meta cites AA (Beam self-report), not whole-problem accuracy. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

61% · raw 61 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Muse internal bash/browser tools with HF/Kaggle/PapersWithCode/LMSYS blocked; question subset unpublished. Peers from self-reports; not proof of identical tool or task budgets. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

89% · raw 89 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 198 questions; Meta cites AA (Beam self-report). As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

60% · raw 60 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 113 tasks/91 repos/five languages; Muse Code for Muse (AA-cited); GLM mini-swe-agent, Kimi Code, Beam/Mistral harness unspecified. Preferred harnesses differ. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

58% · raw 58 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 11 repositories/four languages; 2GB/2CPU sandbox, rubric judge. Muse Code for Muse, GLM OpenCode, Kimi Code; AA-cited. Beam/Mistral self-reports. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

76.4% · raw 76.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 89 tasks; AA-cited Terminus 2/E2B, Beam self-report with harness unspecified. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

50% · raw 50 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 100 questions, pass@5 (at least one correct of five), not pass@1. Tools enabled; GPT-OSS-120B high/container judge. Kimi self-report. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

80.2% · raw 80.2 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 388 questions (135 multiple-choice/253 free-form), mean accuracy over five runs for Muse; tools enabled, GPT-OSS-120B high judge. Kimi self-report. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

73.6% · raw 73.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Tools enabled, 2,024 bilingual questions, five-run mean; Meta internal runs including Kimi. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

49.4% · raw 49.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Tools enabled; 3,000-question/eight-category subset excluding Notable People, GPT-OSS-120B high judge; Kimi self-report. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

68.3% · raw 68.3 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Official main 1,012-video violation-of-expectation accuracy; Meta internal runs including Kimi. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

61.5% · raw 61.5 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: 0-shot full 3,200 questions/800 videos, grouped scoring, subtitles in prompt/no audio; Meta internal runs including Kimi. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

5.8% · raw 5.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Scale private split cited by Meta; ARS/ORS retained as separate quantities; lower is better. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

17.4% · raw 17.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Scale private split cited by Meta; ARS/ORS retained as separate quantities; lower is better. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

0.5% · raw 0.5 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Internal 202 behaviors, five attack conditions; mean union-broken ASRs with/without developer prompts; lower is better. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

99.2% · raw 99.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-15

Harness: Adaptive attacks, no system safeguards; internal/public-suite aggregate refusal. Effort and denominator unpublished. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

94.8% · raw 94.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-15

Harness: Adaptive attacks, no system safeguards; internal/public-suite aggregate refusal. Effort and denominator unpublished. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

98.1% · raw 98.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-15

Harness: Source-specific safety assessment; split/scenarios/sample size/effort unpublished. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

0% · raw 0 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-15

Harness: Source-specific safety assessment; split/scenarios/sample size/effort unpublished. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

3% · raw 3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-15

Meta reports a 3% end-to-end speedup on AMD MI350 with custom auto-tunable Helion GPU kernels in SGLang. Exact baseline, workload, runtime version and weight precision unpublished. Relative improvement only, not API speed or a controlled cross-model comparison. Kernel-level 1.03–3.95x range is heterogeneous context, not an exact per-kernel measurement.

60.665% · raw 60.665 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1612.347,"stderr":2.056,"cost_per_test":2.758047,"token_totals":{"input_tokens":3043998798,"output_tokens":14268191,"reasoning_tokens":7831933,"cache_read_tokens":2388431856,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

$2.758047/task · raw 2.758047 USD/task

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1612.347,"stderr":2.056,"cost_per_test":2.758047,"token_totals":{"input_tokens":3043998798,"output_tokens":14268191,"reasoning_tokens":7831933,"cache_read_tokens":2388431856,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

1612.347 s · raw 1612.347 s

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1612.347,"stderr":2.056,"cost_per_test":2.758047,"token_totals":{"input_tokens":3043998798,"output_tokens":14268191,"reasoning_tokens":7831933,"cache_read_tokens":2388431856,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

48.957% · raw 48.957 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1612.347,"stderr":2.204,"cost_per_test":2.758047,"token_totals":{"input_tokens":3043998798,"output_tokens":14268191,"reasoning_tokens":7831933,"cache_read_tokens":2388431856,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

75.291% · raw 75.291 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1581.951,"stderr":5.479,"cost_per_test":2.006504,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

75.883% · raw 75.883 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":894.853,"stderr":5.344,"cost_per_test":1.160885,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

56.642% · raw 56.642 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1287.19,"stderr":7.026,"cost_per_test":4.555468,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

54.442% · raw 54.442 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2030.108,"stderr":6.588,"cost_per_test":1.599423,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

39.255% · raw 39.255 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2050.598,"stderr":7.053,"cost_per_test":3.945511,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

62.995% · raw 62.995 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1335.692,"stderr":6.801,"cost_per_test":2.51918,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

81.263% · raw 81.263 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1037.852,"stderr":5.468,"cost_per_test":1.221736,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

76.791% · raw 76.791 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1369.397,"stderr":5.676,"cost_per_test":1.846256,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

23.420% · raw 23.42 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. FAB v2 harness on expert-reviewed FABv2.1 questions/rubrics, revised judge jury and hedged-answer prompt. Partial-credit accuracy differs from all-pass. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3287.51,"stderr":5.749,"cost_per_test":6.232657,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

61.983% · raw 61.983 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

$5.348821/task · raw 5.348821 USD/task

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

5078.614 s · raw 5078.614 s

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

61.805% · raw 61.805 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5739.054,"stderr":3.943,"cost_per_test":7.009169,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

61.720% · raw 61.72 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":4418.173,"stderr":2.699,"cost_per_test":3.688473,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

48.325% · raw 48.325 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5821.099,"stderr":5.029,"cost_per_test":7.431962,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

49.913% · raw 49.913 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5120.532,"stderr":7.255,"cost_per_test":4.653132,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

71.133% · raw 71.133 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5014.642,"stderr":5.319,"cost_per_test":3.855837,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

68.645% · raw 68.645 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":4388.685,"stderr":5.159,"cost_per_test":4.195928,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

56.044% · raw 56.044 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":4497.271,"stderr":6.916,"cost_per_test":6.33156,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

79.535% · raw 79.535 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5754.881,"stderr":5.952,"cost_per_test":6.145752,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

60.284% · raw 60.284 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5020.414,"stderr":3.136,"cost_per_test":4.421871,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

48.179% · raw 48.179 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

48.471% · raw 48.471 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

46.972% · raw 46.972 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

52.855% · raw 52.855 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

72.516% · raw 72.516 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

69.751% · raw 69.751 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

69.456% · raw 69.456 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

67.834% · raw 67.834 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

50.354% · raw 50.354 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

61.735% · raw 61.735 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

84.357% · raw 84.357 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

74.713% · raw 74.713 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

63.541% · raw 63.541 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

57.027% · raw 57.027 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

55.892% · raw 55.892 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

81.199% · raw 81.199 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

68.697% · raw 68.697 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Seven equally weighted category scores combining Template and Scratch. GPT-6 Luna Scratch judge and Template sheet-name scoring fixes; not comparable with v1.0. Partial credit is not client-ready quality. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5078.614,"stderr":2.704,"cost_per_test":5.348821,"token_totals":{"input_tokens":973042320,"output_tokens":8495845,"reasoning_tokens":4888586,"cache_read_tokens":642573184,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

26.893% · raw 26.893 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5021.245,"stderr":3.229,"cost_per_test":3.767449,"token_totals":{"input_tokens":1717177941,"output_tokens":11784542,"reasoning_tokens":5997374,"cache_read_tokens":1500265728,"cache_write_tokens":0},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

$3.767449/task · raw 3.767449 USD/task

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5021.245,"stderr":3.229,"cost_per_test":3.767449,"token_totals":{"input_tokens":1717177941,"output_tokens":11784542,"reasoning_tokens":5997374,"cache_read_tokens":1500265728,"cache_write_tokens":0},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

5021.245 s · raw 5021.245 s

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5021.245,"stderr":3.229,"cost_per_test":3.767449,"token_totals":{"input_tokens":1717177941,"output_tokens":11784542,"reasoning_tokens":5997374,"cache_read_tokens":1500265728,"cache_write_tokens":0},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

15.024% · raw 15.024 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5660.496,"stderr":1.774,"cost_per_test":4.395183,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

62.500% · raw 62.5 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3103.489,"stderr":11.767,"cost_per_test":1.884248,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

2.398 points/5 · raw 2.398 points/5

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: Pi. Pi agent harness, revised hidden tests, full 120 CLI + 10 COBOL tasks; 75% CLI and 25% COBOL weighting. Distinct from Index selected 50 CLI +10 COBOL subset and old mini-SWE-agent v1. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5021.245,"stderr":0.039,"cost_per_test":3.767449,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

26.321% · raw 26.321 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":701.825,"stderr":2.653,"cost_per_test":0.883559,"token_totals":{"input_tokens":413690950,"output_tokens":3571385,"reasoning_tokens":1667757,"cache_read_tokens":329680688,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

$0.883559/task · raw 0.883559 USD/task

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":701.825,"stderr":2.653,"cost_per_test":0.883559,"token_totals":{"input_tokens":413690950,"output_tokens":3571385,"reasoning_tokens":1667757,"cache_read_tokens":329680688,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

701.825 s · raw 701.825 s

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":701.825,"stderr":2.653,"cost_per_test":0.883559,"token_totals":{"input_tokens":413690950,"output_tokens":3571385,"reasoning_tokens":1667757,"cache_read_tokens":329680688,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

26.744% · raw 26.744 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":995.717,"stderr":5.404,"cost_per_test":2.528872,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

42.333% · raw 42.333 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":430.403,"stderr":7.591,"cost_per_test":0.193793,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

28.000% · raw 28 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":760.495,"stderr":6.368,"cost_per_test":0.288385,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

38.667% · raw 38.667 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":339.074,"stderr":7.904,"cost_per_test":0.15679,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

11.667% · raw 11.667 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":848.017,"stderr":4.844,"cost_per_test":0.290359,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

10.333% · raw 10.333 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":826.163,"stderr":4.971,"cost_per_test":1.13869,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

52.066% · raw 52.066 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":701.825,"stderr":2.387,"cost_per_test":0.883559,"token_totals":{"input_tokens":413690950,"output_tokens":3571385,"reasoning_tokens":1667757,"cache_read_tokens":329680688,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

43.596% · raw 43.596 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":995.717,"stderr":5.271,"cost_per_test":2.528872,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

60.287% · raw 60.287 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":430.403,"stderr":6.065,"cost_per_test":0.193793,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

47.221% · raw 47.221 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":760.495,"stderr":5.768,"cost_per_test":0.288385,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

68.911% · raw 68.911 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":339.074,"stderr":5.042,"cost_per_test":0.15679,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

50.385% · raw 50.385 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":848.017,"stderr":5.212,"cost_per_test":0.290359,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

45.667% · raw 45.667 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. Private193-item tax suite. All-pass accuracy and must-pass weighted partial credit are separate; both multiply by citation factor0.7+0.3*citation quality. No inferred tax professional reliability. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":826.163,"stderr":6.659,"cost_per_test":1.13869,"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":null}

76.187% · raw 76.187 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. Independent model-plus-harness web-application evaluation. Preserve OpenHands versus other agent variants; source costs/runtime are task observations, not API tariffs. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1869.172,"stderr":3.317,"cost_per_test":1.655609,"token_totals":{"input_tokens":225685852,"output_tokens":2085879,"reasoning_tokens":952456,"cache_read_tokens":189265312,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":"OpenHands"}

$1.655609/task · raw 1.655609 USD/task

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. Independent model-plus-harness web-application evaluation. Preserve OpenHands versus other agent variants; source costs/runtime are task observations, not API tariffs. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1869.172,"stderr":3.317,"cost_per_test":1.655609,"token_totals":{"input_tokens":225685852,"output_tokens":2085879,"reasoning_tokens":952456,"cache_read_tokens":189265312,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":"OpenHands"} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

1869.172 s · raw 1869.172 s

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: OpenHands. Independent model-plus-harness web-application evaluation. Preserve OpenHands versus other agent variants; source costs/runtime are task observations, not API tariffs. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1869.172,"stderr":3.317,"cost_per_test":1.655609,"token_totals":{"input_tokens":225685852,"output_tokens":2085879,"reasoning_tokens":952456,"cache_read_tokens":189265312,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"Together AI","harness":"OpenHands"} Source score standard error describes the benchmark quality score, not uncertainty in cost or latency.

6.03 acronyms/response · raw 6.03 acronyms/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Acronyms: Two or more capital letters in a row (AI, USA) or dotted forms (U.S.). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 6.03.

13.80 words/sentence · raw 13.8 words/sentence

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Avg. sentence length: Words divided by sentences. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 13.8.

12.50 bold spans/response · raw 12.5 bold spans/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Bold text: Uses of bold: each unbroken run of bold text counts once, so a single bolded word and a fully bolded sentence both count as 1. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 12.5.

5050 characters/response · raw 5050 characters/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Characters: Characters in the prose, spaces included, after code and URLs are removed. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 5,050.

19.00 colons/response · raw 19 colons/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Colons: Colons. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 19.0.

19.30 contractions/response · raw 19.3 contractions/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Contractions: Contractions such as “don’t”, “you’re” and “it’s”; possessives such as “John’s” do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 19.3.

0.36 ellipses/response · raw 0.36 ellipses/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Ellipses: Three periods (...) or the ellipsis character (…). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.36.

5.56 em dashes/response · raw 5.56 em dashes/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Em dashes: The em dash character (—); double hyphens do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 5.56.

0.01 emojis/response · raw 0.01 emojis/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Emojis: Each emoji as the reader sees it. An emoji built from several characters, such as a family (👨‍👩‍👧), still counts as one emoji. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.01.

26% · raw 26 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Ends with a question: Whether the answer’s last character is a question mark, reported as the share of responses that do. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 26%.

0.33 exclamations/response · raw 0.33 exclamations/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Exclamations: Exclamation marks. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.33.

4.30 headings/response · raw 4.3 headings/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Headings: Headings of any level. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 4.30.

5.41 italic spans/response · raw 5.41 italic spans/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Italics: Uses of italics: each unbroken run of italic text counts once, so a single italicized word and a fully italicized sentence both count as 1. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 5.41.

4.38 lists/response · raw 4.38 lists/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Lists: Top-level bulleted or numbered lists; nested sub-lists do not count separately. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 4.38.

22.50 paragraphs/response · raw 22.5 paragraphs/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Paragraphs: Paragraphs of prose; headings, code and the items of a single-spaced list do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 22.5.

3.32 parenthesis pairs/response · raw 3.32 parenthesis pairs/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Parentheses: Closed pairs of parentheses, nested pairs included; unmatched parentheses do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 3.32.

8.37 question marks/response · raw 8.37 question marks/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Questions: Question marks. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 8.37.

15.10 quotation marks/response · raw 15.1 quotation marks/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Quotes: Quotation marks of any style, such as straight, curly, « » and „ “; apostrophes do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 15.1.

63.20 sentences/response · raw 63.2 sentences/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Sentences: Sentences found by a rule-based splitter that handles abbreviations such as “Dr.”. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 63.2.

0.05 tables/response · raw 0.05 tables/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Tables: Tables. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.05.

809 words/response · raw 809 words/response

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Words: Tokens that contain a letter or digit; each Chinese, Japanese or Korean character counts as one word (but all questions are in English). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 809.

26 percentile · raw 26 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Direct, confident positions and constructive pushback. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 26.

49 percentile · raw 49 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Rich, varied, well-crafted language. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 49.

97 percentile · raw 97 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Creative, exploratory, open-ended thinking. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 97.

89 percentile · raw 89 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Practical depth, evidence, actionable content. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 89.

29 percentile · raw 29 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Well-structured, scannable, answer-first. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 29.

86 percentile · raw 86 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Use of filler and generic language. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 86.

55 percentile · raw 55 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Truthful, acknowledges uncertainty, non-sycophantic. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 55.

86 percentile · raw 86 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Friendly, humorous, sincere, emotionally attuned. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 86.

92 percentile · raw 92 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Actionable Guidance writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 92.

43 percentile · raw 43 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Admits Uncertainty writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 43.

29 percentile · raw 29 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Answer First writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 29.

60 percentile · raw 60 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Bottom-Line Summary writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 60.

18 percentile · raw 18 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Confident writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 18.

72 percentile · raw 72 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Constructive Pushback writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 72.

80 percentile · raw 80 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Creative writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 80.

4 percentile · raw 4 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Direct writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 4.

94 percentile · raw 94 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Disclaims Assumptions writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 94.

58 percentile · raw 58 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Eloquent writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 58.

83 percentile · raw 83 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Emotionally Intelligent writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 83.

72 percentile · raw 72 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Evidence-Based writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 72.

92 percentile · raw 92 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Flags Nuances writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 92.

97 percentile · raw 97 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Friendly writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 97.

80 percentile · raw 80 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Humorous writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 80.

66 percentile · raw 66 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Information Dense writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 66.

60 percentile · raw 60 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Lexically Varied writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 60.

7 percentile · raw 7 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Literate writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 7.

97 percentile · raw 97 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Metaphorical writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 97.

38 percentile · raw 38 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Natural Sounding writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 38.

97 percentile · raw 97 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Open-Ended writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 97.

75 percentile · raw 75 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Opinionated writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 75.

66 percentile · raw 66 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Pragmatic writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 66.

89 percentile · raw 89 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Resource-Specific writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 89.

38 percentile · raw 38 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Scannable writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 38.

35 percentile · raw 35 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Sincere writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 35.

86 percentile · raw 86 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Slop writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 86.

32 percentile · raw 32 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Structured writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 32.

75 percentile · raw 75 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Sycophantic writing trait, judged pairwise among responses to the same questions. Native trait direction: higher means more sycophancy; only its contribution to Transparency is inverted. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 75.

49 percentile · raw 49 percentile

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-19

Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Truthful writing trait, judged pairwise among responses to the same questions. A judged writing/personality trait in this question population, not a general factual-accuracy benchmark. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 49.

1401 · raw 1401 Elo

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Manually observed public GDPval-AA v2.1 leaderboard table, full 298-configuration pool checked 20 October 2026. Exact source configuration: Muse Spark Open (Xhigh); creator: Meta; source rank 75; source release-month label Oct 2026 is model release metadata, not execution date. Published rounded native Elo 1401; literal displayed CI -19 / +19. Native Elo is not a percentage and is not the normalized (Elo−500)/2000 chart quantity. This versioned metric is separate from older generic GDPval-AA Elo and vendor GDPval/GDPval-AA results; no old evidence is overwritten. Execution date, inference precision, unprinted effort budgets and task costs are unpublished in this captured table. Elo is relative to the dated comparison pool; drift from an earlier observation does not prove a new evaluation run. Headline: highest published native Elo for this exact approved model; tied values prefer existing featured effort then source order.

38.9 · raw 38.8666662997805 index

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished.

24.7 · raw 24.65 index

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished.

39.3% · raw 39.266666666666666 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

24.1% · raw 24.06695938529089 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

1401.1 · raw 1401.12 Elo

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished.

76.4% · raw 76.4044943820225 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

8.1% · raw 8.08080808080808 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

58.4% · raw 58.449074074074105 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

78.3% · raw 78.3333333333333 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

43.9% · raw 43.8832252085264 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

89% · raw 88.98989898989899 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

14.9% · raw 14.857142857142899 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

18.6% · raw 18.6 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

49.7% · raw 49.70811827313194 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained. Partial reward, not strict all-assertion pass rate.

80.9% · raw 80.9248554913295 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

42.5 · raw 42.5019485585335 index

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished.

42.7 · raw 42.6714014343989 index

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished.

44.5 · raw 44.5469915351241 index

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished.

40.4 · raw 40.4428764219716 index

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished.

51 · raw 50.9916704896509 index

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished.

1279.3 · raw 1279.25 Elo

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. 95% CI 1269.04 to 1290.21.

1256.1 · raw 1256.07 Elo

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. 95% CI 1236.01 to 1276.79.

1301.5 · raw 1301.46 Elo

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. 95% CI 1282.73 to 1321.89.

45.7% · raw 45.656565656565654 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. Published fraction ×100; raw fraction retained.

102 tokens/s · raw 102.036199790718 tokens/s

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. AA source: {'type': 'median'}; API prompt/harness observation, not local inference or OpenRouter endpoint speed.

$0.844/task · raw 0.8435187802958677 USD/task

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-16

Independent AA exact model page. Published xhigh effort. Index v4.3.2 where stated. Inference precision and execution dates unpublished. AA-reported modeled evaluation cost per task; not subscription bills or token rates.