ExamenosMuse Glimmer 30B
Explore Muse Glimmer 30B from Meta: published specifications, source-linked vendor benchmarks, independent evaluator coverage and recorded pricing when available.
Compare Muse Glimmer 30B with other models →Compare subscription plansExplore data coverage
Pricing · OpenRouter
Dated cached OpenRouter rates in USD per 1M tokens. Open the dashboard for live enhancements. Per-metric endpoint minima can refer to different providers; they are not a guaranteed combined rate from one endpoint.
| Tier | Input / 1M | Output / 1M | Cached input / 1M | Cache write / 1M | Date & source |
|---|---|---|---|---|---|
| Default | $0.3 | $1.2 | $0.04 | — | 2026-10-21 · OpenRouter source |
Recorded pricing notes
min-healthy endpoint minima 2026-10-21; min-healthy endpoint minima 2026-10-21
Official / vendor benchmarks
Default headline records. Own-vendor, peer-vendor and third-party provenance remain visible in evidence; configurations may differ.
| Benchmark / evaluator | Headline score | Evidence |
|---|---|---|
| Tau3 Banking (Meta AA cited) | 23.5%high effort | All 2 recorded results & sources23.5% · raw 23.5 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. AA-sourced by Meta; not a current independently verified evaluator result. https://arxiv.org/html/2610.24628v1#A1.T526% · raw 26 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. AA-sourced by Meta; not a current independently verified evaluator result. https://arxiv.org/html/2610.24628v1#A1.T5 |
| WildClawBench | 47.6%high effort | All 1 recorded result & sources47.6% · raw 47.6 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 60 tasks, OpenClaw Docker, GPT-5.4 / task-specific grading,3 runs; mean partial reward. https://arxiv.org/html/2610.24628v1#A1.T5 |
| GAIA2 OpenClaw | 43.3%high effort | All 1 recorded result & sources43.3% · raw 43.3 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 800 tasks / 5 capabilities, OpenClaw, gpt-oss-120b judge,3 runs; mean per-sample success. https://arxiv.org/html/2610.24628v1#A1.T5 |
| GDPval-AA v2 (Meta AA cited Elo) | 953high effort | All 2 recorded results & sources953 · raw 953 Elo Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 220 tasks / 44 occupations; Stirrup E2B,250 turns; native Elo in August AA v2 pool; not current v2.1. https://arxiv.org/html/2610.24628v1#A1.T5952 · raw 952 Elo Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 220 tasks / 44 occupations; Stirrup E2B,250 turns; native Elo in August AA v2 pool; not current v2.1. https://arxiv.org/html/2610.24628v1#A1.T5 |
| SciCode subtask (Meta pre-v4.2 AA) | 43.6%high effort | All 2 recorded results & sources43.6% · raw 43.6 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 288 subproblems with background, no tools; AA values from before September2026 v4.2 change. https://arxiv.org/html/2610.24628v1#A1.T539.2% · raw 39.2 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 288 subproblems with background, no tools; AA values from before September2026 v4.2 change. https://arxiv.org/html/2610.24628v1#A1.T5 |
| OSWorld-Verified (Meta361) | 65.9%high effort | All 4 recorded results & sources58.7% · raw 58.7 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 361 tasks, excludes 8 Google Drive tasks, 200 steps, 1920×1080 screenshots, 19-image history; Claude computer_20251124 for Glimmer, Gemini interface for Gemma;4 attempts. https://arxiv.org/html/2610.24628v1#A1.T566.6% · raw 66.6 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 361 tasks, excludes 8 Google Drive tasks, 200 steps, 1920×1080 screenshots, 19-image history; Claude computer_20251124 for Glimmer, Gemini interface for Gemma;4 attempts. https://arxiv.org/html/2610.24628v1#A1.T565.9% · raw 65.9 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 361 tasks, excludes 8 Google Drive tasks, 200 steps, 1920×1080 screenshots, 19-image history; Claude computer_20251124 for Glimmer, Gemini interface for Gemma;4 attempts. https://arxiv.org/html/2610.24628v1#A1.T569.7% · raw 69.7 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 361 tasks, excludes 8 Google Drive tasks, 200 steps, 1920×1080 screenshots, 19-image history; Claude computer_20251124 for Glimmer, Gemini interface for Gemma;4 attempts. https://arxiv.org/html/2610.24628v1#A1.T5 |
| SWE-bench Verified (Meta bash) | 76%high effort | All 4 recorded results & sources74.6% · raw 74.6 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 500 tasks, bash + file tool, 4 runs, mean pass rate. https://arxiv.org/html/2610.24628v1#A1.T575.2% · raw 75.2 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 500 tasks, bash + file tool, 4 runs, mean pass rate. https://arxiv.org/html/2610.24628v1#A1.T576% · raw 76 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 500 tasks, bash + file tool, 4 runs, mean pass rate. https://arxiv.org/html/2610.24628v1#A1.T575.3% · raw 75.3 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 500 tasks, bash + file tool, 4 runs, mean pass rate. https://arxiv.org/html/2610.24628v1#A1.T5 |
| SWE-bench Pro (Meta731) | 51.2%high effort | All 4 recorded results & sources50.5% · raw 50.5 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. Original 731 tasks, not refined Qwen subset; bash + file tool, 4 runs. https://arxiv.org/html/2610.24628v1#A1.T550.9% · raw 50.9 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. Original 731 tasks, not refined Qwen subset; bash + file tool, 4 runs. https://arxiv.org/html/2610.24628v1#A1.T551.2% · raw 51.2 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. Original 731 tasks, not refined Qwen subset; bash + file tool, 4 runs. https://arxiv.org/html/2610.24628v1#A1.T551.6% · raw 51.6 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. Original 731 tasks, not refined Qwen subset; bash + file tool, 4 runs. https://arxiv.org/html/2610.24628v1#A1.T5 |
| TauBench Airline (Meta50) | 78%high effort | All 4 recorded results & sources79% · raw 79 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 50 airline tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T577% · raw 77 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 50 airline tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T578% · raw 78 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 50 airline tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T577.5% · raw 77.5 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 50 airline tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T5 |
| TauBench Retail (Meta110) | 87.1%high effort | All 4 recorded results & sources86% · raw 86 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 110 retail tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T588.4% · raw 88.4 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 110 retail tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T587.1% · raw 87.1 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 110 retail tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T586.8% · raw 86.8 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 110 retail tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T5 |
| TauBench Telecom (Meta114) | 94.1%high effort | All 4 recorded results & sources91.6% · raw 91.6 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 114 telecom tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T592.5% · raw 92.5 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 114 telecom tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T594.1% · raw 94.1 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 114 telecom tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T592.1% · raw 92.1 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 114 telecom tasks,4 runs, database/assertion success. https://arxiv.org/html/2610.24628v1#A1.T5 |
| Terminal-Bench2.1 Terminus2 (Meta AA cited) | 51.7%high effort | All 2 recorded results & sources51.7% · raw 51.7 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 89 tasks, Terminus2/E2B; Meta cites AA. https://arxiv.org/html/2610.24628v1#A1.T554.3% · raw 54.3 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 89 tasks, Terminus2/E2B; Meta cites AA. https://arxiv.org/html/2610.24628v1#A1.T5 |
| LiveCodeBench Pro 2025Q2 C++ | 64.1%high effort | All 4 recorded results & sources36.2% · raw 36.2 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2025Q2 Codeforces subset, C++, complete hidden checks,8 attempts/problem pass@1. https://arxiv.org/html/2610.24628v1#A1.T555.8% · raw 55.8 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2025Q2 Codeforces subset, C++, complete hidden checks,8 attempts/problem pass@1. https://arxiv.org/html/2610.24628v1#A1.T564.1% · raw 64.1 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2025Q2 Codeforces subset, C++, complete hidden checks,8 attempts/problem pass@1. https://arxiv.org/html/2610.24628v1#A1.T566.8% · raw 66.8 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2025Q2 Codeforces subset, C++, complete hidden checks,8 attempts/problem pass@1. https://arxiv.org/html/2610.24628v1#A1.T5 |
| PinchBench (Meta24 diagnostic) | 90.5%high effort | All 4 recorded results & sources89% · raw 89 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 24 tasks, bash/files/search/fetch, gpt-oss-120b+deterministic judge,4 runs. Source describes small free-form set as diagnostic. https://arxiv.org/html/2610.24628v1#A1.T590.9% · raw 90.9 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 24 tasks, bash/files/search/fetch, gpt-oss-120b+deterministic judge,4 runs. Source describes small free-form set as diagnostic. https://arxiv.org/html/2610.24628v1#A1.T590.5% · raw 90.5 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 24 tasks, bash/files/search/fetch, gpt-oss-120b+deterministic judge,4 runs. Source describes small free-form set as diagnostic. https://arxiv.org/html/2610.24628v1#A1.T590.7% · raw 90.7 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 24 tasks, bash/files/search/fetch, gpt-oss-120b+deterministic judge,4 runs. Source describes small free-form set as diagnostic. https://arxiv.org/html/2610.24628v1#A1.T5 |
| MCP Atlas (Meta public500) | 75.5%high effort | All 4 recorded results & sources72.9% · raw 72.9 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 500 public tasks, 20+ MCP servers; Gemini 2.5 Pro judge; score > 0.75 for task pass;4 runs. https://arxiv.org/html/2610.24628v1#A1.T576.3% · raw 76.3 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 500 public tasks, 20+ MCP servers; Gemini 2.5 Pro judge; score > 0.75 for task pass;4 runs. https://arxiv.org/html/2610.24628v1#A1.T575.5% · raw 75.5 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 500 public tasks, 20+ MCP servers; Gemini 2.5 Pro judge; score > 0.75 for task pass;4 runs. https://arxiv.org/html/2610.24628v1#A1.T577.4% · raw 77.4 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 500 public tasks, 20+ MCP servers; Gemini 2.5 Pro judge; score > 0.75 for task pass;4 runs. https://arxiv.org/html/2610.24628v1#A1.T5 |
| DeepSearchQA F1 (Meta900) | 74.6%high effort | All 4 recorded results & sources68.4% · raw 68.4 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 900 questions; search/open/find; gpt-oss-120b extracted answer-set F1;4 runs. https://arxiv.org/html/2610.24628v1#A1.T572.9% · raw 72.9 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 900 questions; search/open/find; gpt-oss-120b extracted answer-set F1;4 runs. https://arxiv.org/html/2610.24628v1#A1.T574.6% · raw 74.6 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 900 questions; search/open/find; gpt-oss-120b extracted answer-set F1;4 runs. https://arxiv.org/html/2610.24628v1#A1.T575.3% · raw 75.3 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 900 questions; search/open/find; gpt-oss-120b extracted answer-set F1;4 runs. https://arxiv.org/html/2610.24628v1#A1.T5 |
| AIME2026 (Meta avg10) | 94.7%high effort | All 4 recorded results & sources69.7% · raw 69.7 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 30 questions, 10 runs mean accuracy; peer values self-reported. https://arxiv.org/html/2610.24628v1#A1.T589% · raw 89 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 30 questions, 10 runs mean accuracy; peer values self-reported. https://arxiv.org/html/2610.24628v1#A1.T594.7% · raw 94.7 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 30 questions, 10 runs mean accuracy; peer values self-reported. https://arxiv.org/html/2610.24628v1#A1.T593% · raw 93 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 30 questions, 10 runs mean accuracy; peer values self-reported. https://arxiv.org/html/2610.24628v1#A1.T5 |
| GPQA Diamond (Meta AA cited) | 83.5%high effort | All 2 recorded results & sources83.5% · raw 83.5 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 198 questions, no tools; Meta cites AA. https://arxiv.org/html/2610.24628v1#A1.T585.6% · raw 85.6 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 198 questions, no tools; Meta cites AA. https://arxiv.org/html/2610.24628v1#A1.T5 |
| ARC-AGI (Meta400, version unspecified) | 66.9%high effort | All 4 recorded results & sources34.6% · raw 34.6 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 400 questions, serial generation/verification, 16 iterations/question, no test-time tuning; version notpublished. https://arxiv.org/html/2610.24628v1#A1.T558.8% · raw 58.8 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 400 questions, serial generation/verification, 16 iterations/question, no test-time tuning; version notpublished. https://arxiv.org/html/2610.24628v1#A1.T566.9% · raw 66.9 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 400 questions, serial generation/verification, 16 iterations/question, no test-time tuning; version notpublished. https://arxiv.org/html/2610.24628v1#A1.T567.9% · raw 67.9 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 400 questions, serial generation/verification, 16 iterations/question, no test-time tuning; version notpublished. https://arxiv.org/html/2610.24628v1#A1.T5 |
| HLE Text (Meta AA cited2158) | 22%high effort | All 2 recorded results & sources22% · raw 22 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,158 text-only questions; no tools; Meta cites AA. https://arxiv.org/html/2610.24628v1#A1.T523.4% · raw 23.4 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,158 text-only questions; no tools; Meta cites AA. https://arxiv.org/html/2610.24628v1#A1.T5 |
| IFBench (Meta 294) | 77%high effort | All 4 recorded results & sources67.4% · raw 67.4 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 294 constraints/prompts,4 runs meanpass; exact strict/loose aggregation not restated. https://arxiv.org/html/2610.24628v1#A1.T573.3% · raw 73.3 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 294 constraints/prompts,4 runs meanpass; exact strict/loose aggregation not restated. https://arxiv.org/html/2610.24628v1#A1.T577% · raw 77 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 294 constraints/prompts,4 runs meanpass; exact strict/loose aggregation not restated. https://arxiv.org/html/2610.24628v1#A1.T577.1% · raw 77.1 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 294 constraints/prompts,4 runs meanpass; exact strict/loose aggregation not restated. https://arxiv.org/html/2610.24628v1#A1.T5 |
| SkillsBench (Meta86 weighted reward) | 44.3%high effort | All 4 recorded results & sources40% · raw 40 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 86 tasks; bash/image/browser; hidden pytest weighted fraction;4 attempts. https://arxiv.org/html/2610.24628v1#A1.T543.6% · raw 43.6 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 86 tasks; bash/image/browser; hidden pytest weighted fraction;4 attempts. https://arxiv.org/html/2610.24628v1#A1.T544.3% · raw 44.3 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 86 tasks; bash/image/browser; hidden pytest weighted fraction;4 attempts. https://arxiv.org/html/2610.24628v1#A1.T543.2% · raw 43.2 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 86 tasks; bash/image/browser; hidden pytest weighted fraction;4 attempts. https://arxiv.org/html/2610.24628v1#A1.T5 |
| CharXiv Reasoning (Meta 1000) | 78.8%high effort | All 4 recorded results & sources75.4% · raw 75.4 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,000 validation chart questions, gpt-oss-120b judge,4 runs; Qwen peer self-reported. https://arxiv.org/html/2610.24628v1#A1.T577.9% · raw 77.9 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,000 validation chart questions, gpt-oss-120b judge,4 runs; Qwen peer self-reported. https://arxiv.org/html/2610.24628v1#A1.T578.8% · raw 78.8 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,000 validation chart questions, gpt-oss-120b judge,4 runs; Qwen peer self-reported. https://arxiv.org/html/2610.24628v1#A1.T578.6% · raw 78.6 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,000 validation chart questions, gpt-oss-120b judge,4 runs; Qwen peer self-reported. https://arxiv.org/html/2610.24628v1#A1.T5 |
| MMMU-Pro (Meta AA cited) | 74%high effort | All 1 recorded result & sources74% · raw 74 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,730 questions; Meta cites AA. https://arxiv.org/html/2610.24628v1#A1.T5 |
| OmniDocBench1.5 (Meta modified two-component) | 75.8%high effort | All 4 recorded results & sources75.3% · raw 75.3 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,355 prompts, modified two-component weighted text/formulas + tables; Hungarian matching; excludes separate formula CDM;4 runs. Distinct from official three-component v1.5. https://arxiv.org/html/2610.24628v1#A1.T575.7% · raw 75.7 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,355 prompts, modified two-component weighted text/formulas + tables; Hungarian matching; excludes separate formula CDM;4 runs. Distinct from official three-component v1.5. https://arxiv.org/html/2610.24628v1#A1.T575.8% · raw 75.8 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,355 prompts, modified two-component weighted text/formulas + tables; Hungarian matching; excludes separate formula CDM;4 runs. Distinct from official three-component v1.5. https://arxiv.org/html/2610.24628v1#A1.T575.8% · raw 75.8 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,355 prompts, modified two-component weighted text/formulas + tables; Hungarian matching; excludes separate formula CDM;4 runs. Distinct from official three-component v1.5. https://arxiv.org/html/2610.24628v1#A1.T5 |
| ScreenSpot-Pro (Meta iterative crop) | 75.4%high effort | All 4 recorded results & sources75.1% · raw 75.1 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,581 examples; iterative Python crop, grid zoom,10 verification rounds, 3 restarts;4 attempts. Distinct from one-shot grounding. https://arxiv.org/html/2610.24628v1#A1.T575.4% · raw 75.4 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,581 examples; iterative Python crop, grid zoom,10 verification rounds, 3 restarts;4 attempts. Distinct from one-shot grounding. https://arxiv.org/html/2610.24628v1#A1.T575.4% · raw 75.4 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,581 examples; iterative Python crop, grid zoom,10 verification rounds, 3 restarts;4 attempts. Distinct from one-shot grounding. https://arxiv.org/html/2610.24628v1#A1.T575.7% · raw 75.7 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 1,581 examples; iterative Python crop, grid zoom,10 verification rounds, 3 restarts;4 attempts. Distinct from one-shot grounding. https://arxiv.org/html/2610.24628v1#A1.T5 |
| ScreenSpot v2 (Meta no tools) | 93.3%high effort | All 4 recorded results & sources93.2% · raw 93.2 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. No crop tools; normalized coordinates, point-in-box4 attempts. https://arxiv.org/html/2610.24628v1#A1.T592.9% · raw 92.9 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. No crop tools; normalized coordinates, point-in-box4 attempts. https://arxiv.org/html/2610.24628v1#A1.T593.3% · raw 93.3 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. No crop tools; normalized coordinates, point-in-box4 attempts. https://arxiv.org/html/2610.24628v1#A1.T592.5% · raw 92.5 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. No crop tools; normalized coordinates, point-in-box4 attempts. https://arxiv.org/html/2610.24628v1#A1.T5 |
| BEAM128K (Meta400) | 65.1%high effort | All 4 recorded results & sources65% · raw 65 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 400 conversations of 128K tokens; direct, no-tools recall, GPT-5.4 judge,4 attempts. https://arxiv.org/html/2610.24628v1#A1.T565.6% · raw 65.6 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 400 conversations of 128K tokens; direct, no-tools recall, GPT-5.4 judge,4 attempts. https://arxiv.org/html/2610.24628v1#A1.T565.1% · raw 65.1 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 400 conversations of 128K tokens; direct, no-tools recall, GPT-5.4 judge,4 attempts. https://arxiv.org/html/2610.24628v1#A1.T566.2% · raw 66.2 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 400 conversations of 128K tokens; direct, no-tools recall, GPT-5.4 judge,4 attempts. https://arxiv.org/html/2610.24628v1#A1.T5 |
| AA-LCR (Meta pre-v4.2 AA) | 80%high effort | All 2 recorded results & sources80% · raw 80 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 100 questions, no tools; Meta reports pre-release AA values, before September v4.2 changes. https://arxiv.org/html/2610.24628v1#A1.T580% · raw 80 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 100 questions, no tools; Meta reports pre-release AA values, before September v4.2 changes. https://arxiv.org/html/2610.24628v1#A1.T5 |
| CIMemories violation (Meta2500) | 26.4%high effort | All 4 recorded results & sources33.4% · raw 33.4 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,500 scenarios; inappropriate attribute reveal, Claude Sonnet 4.6 judge,4 runs; lower is better. https://arxiv.org/html/2610.24628v1#A1.T531.1% · raw 31.1 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,500 scenarios; inappropriate attribute reveal, Claude Sonnet 4.6 judge,4 runs; lower is better. https://arxiv.org/html/2610.24628v1#A1.T526.4% · raw 26.4 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,500 scenarios; inappropriate attribute reveal, Claude Sonnet 4.6 judge,4 runs; lower is better. https://arxiv.org/html/2610.24628v1#A1.T525.5% · raw 25.5 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,500 scenarios; inappropriate attribute reveal, Claude Sonnet 4.6 judge,4 runs; lower is better. https://arxiv.org/html/2610.24628v1#A1.T5 |
| CIMemories coverage (Meta2500) | 64.8%high effort | All 4 recorded results & sources66.4% · raw 66.4 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,500 scenarios; essential attribute sharing, Claude Sonnet 4.6 judge,4 runs. https://arxiv.org/html/2610.24628v1#A1.T566.3% · raw 66.3 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,500 scenarios; essential attribute sharing, Claude Sonnet 4.6 judge,4 runs. https://arxiv.org/html/2610.24628v1#A1.T564.8% · raw 64.8 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,500 scenarios; essential attribute sharing, Claude Sonnet 4.6 judge,4 runs. https://arxiv.org/html/2610.24628v1#A1.T563.5% · raw 63.5 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 2,500 scenarios; essential attribute sharing, Claude Sonnet 4.6 judge,4 runs. https://arxiv.org/html/2610.24628v1#A1.T5 |
| PrivacyLens adjusted exposure (Meta493) | 49.2%high effort | All 4 recorded results & sources59.6% · raw 59.6 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 493 prompts; adjusted any-item exposure, conditioned on helpfulness>=2; Claude Sonnet 4.6 judge;4 runs; lower is better. https://arxiv.org/html/2610.24628v1#A1.T553% · raw 53 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 493 prompts; adjusted any-item exposure, conditioned on helpfulness>=2; Claude Sonnet 4.6 judge;4 runs; lower is better. https://arxiv.org/html/2610.24628v1#A1.T549.2% · raw 49.2 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 493 prompts; adjusted any-item exposure, conditioned on helpfulness>=2; Claude Sonnet 4.6 judge;4 runs; lower is better. https://arxiv.org/html/2610.24628v1#A1.T559.6% · raw 59.6 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 493 prompts; adjusted any-item exposure, conditioned on helpfulness>=2; Claude Sonnet 4.6 judge;4 runs; lower is better. https://arxiv.org/html/2610.24628v1#A1.T5 |
| Siren AgentDojo ASR@5 (Meta949) | 28.4%high effort | All 4 recorded results & sources29.9% · raw 29.9 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 949 attack scenarios, 97 benign tasks / 35 malicious tasks; Opus 4.6 attacker up to6 iterations; table label ASR@5 retained; method maximum 6 discrepancy. https://arxiv.org/html/2610.24628v1#A1.T527.9% · raw 27.9 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 949 attack scenarios, 97 benign tasks / 35 malicious tasks; Opus 4.6 attacker up to6 iterations; table label ASR@5 retained; method maximum 6 discrepancy. https://arxiv.org/html/2610.24628v1#A1.T528.4% · raw 28.4 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 949 attack scenarios, 97 benign tasks / 35 malicious tasks; Opus 4.6 attacker up to6 iterations; table label ASR@5 retained; method maximum 6 discrepancy. https://arxiv.org/html/2610.24628v1#A1.T527.8% · raw 27.8 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 949 attack scenarios, 97 benign tasks / 35 malicious tasks; Opus 4.6 attacker up to6 iterations; table label ASR@5 retained; method maximum 6 discrepancy. https://arxiv.org/html/2610.24628v1#A1.T5 |
| Siren AgentDojo utility (Meta949) | 94.2%high effort | All 4 recorded results & sources94.2% · raw 94.2 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. Same Siren 949 scenarios; benign completion under attack; deterministic grading. https://arxiv.org/html/2610.24628v1#A1.T594.2% · raw 94.2 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. Same Siren 949 scenarios; benign completion under attack; deterministic grading. https://arxiv.org/html/2610.24628v1#A1.T594.2% · raw 94.2 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. Same Siren 949 scenarios; benign completion under attack; deterministic grading. https://arxiv.org/html/2610.24628v1#A1.T594.2% · raw 94.2 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. Same Siren 949 scenarios; benign completion under attack; deterministic grading. https://arxiv.org/html/2610.24628v1#A1.T5 |
| OmnilingualGAIA2 target mean (Meta 10 languages) | 60.4%high effort | All 4 recorded results & sources58.4% · raw 58.4 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 10 non-English languages / four core capabilities, four-capability macro pass@1, 3 runs; excludes English anchor. https://arxiv.org/html/2610.24628v1#A1.T560.7% · raw 60.7 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 10 non-English languages / four core capabilities, four-capability macro pass@1, 3 runs; excludes English anchor. https://arxiv.org/html/2610.24628v1#A1.T560.4% · raw 60.4 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 10 non-English languages / four core capabilities, four-capability macro pass@1, 3 runs; excludes English anchor. https://arxiv.org/html/2610.24628v1#A1.T560.5% · raw 60.5 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 10 non-English languages / four core capabilities, four-capability macro pass@1, 3 runs; excludes English anchor. https://arxiv.org/html/2610.24628v1#A1.T5 |
| Linguini whole-problem exact match | 8.8%high effort | All 4 recorded results & sources4.4% · raw 4.4 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 160 problems / 894 questions / 75 low-resource languages, normalized exact match; whole problem must be entirely correct. https://arxiv.org/html/2610.24628v1#A1.T54.4% · raw 4.4 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 160 problems / 894 questions / 75 low-resource languages, normalized exact match; whole problem must be entirely correct. https://arxiv.org/html/2610.24628v1#A1.T58.8% · raw 8.8 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 160 problems / 894 questions / 75 low-resource languages, normalized exact match; whole problem must be entirely correct. https://arxiv.org/html/2610.24628v1#A1.T58.1% · raw 8.1 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 160 problems / 894 questions / 75 low-resource languages, normalized exact match; whole problem must be entirely correct. https://arxiv.org/html/2610.24628v1#A1.T5 |
| Belebele mean accuracy (Meta122 languages) | 87.8%high effort | All 4 recorded results & sources87% · raw 87 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 122 parallel language variants, mean readingcomprehensionaccuracy. https://arxiv.org/html/2610.24628v1#A1.T587.6% · raw 87.6 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 122 parallel language variants, mean readingcomprehensionaccuracy. https://arxiv.org/html/2610.24628v1#A1.T587.8% · raw 87.8 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 122 parallel language variants, mean readingcomprehensionaccuracy. https://arxiv.org/html/2610.24628v1#A1.T587.8% · raw 87.8 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 122 parallel language variants, mean readingcomprehensionaccuracy. https://arxiv.org/html/2610.24628v1#A1.T5 |
| MMLU-ProX (Meta 29 languages) | 80.1%high effort | All 4 recorded results & sources76.9% · raw 76.9 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 29 languages, full test ~12K/language, lm-eval 0-shot, mean pass@1. https://arxiv.org/html/2610.24628v1#A1.T579.3% · raw 79.3 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 29 languages, full test ~12K/language, lm-eval 0-shot, mean pass@1. https://arxiv.org/html/2610.24628v1#A1.T580.1% · raw 80.1 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 29 languages, full test ~12K/language, lm-eval 0-shot, mean pass@1. https://arxiv.org/html/2610.24628v1#A1.T580% · raw 80 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. 29 languages, full test ~12K/language, lm-eval 0-shot, mean pass@1. https://arxiv.org/html/2610.24628v1#A1.T5 |
| WMT24++ chrF++ (Meta110 directions) | 53.4 chrF++high effort | All 4 recorded results & sources53.2 chrF++ · raw 53.2 chrF++ Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,55 into-English + 55 from-English language pairs; source report already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T553.3 chrF++ · raw 53.3 chrF++ Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,55 into-English + 55 from-English language pairs; source report already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T553.4 chrF++ · raw 53.4 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,55 into-English + 55 from-English language pairs; source report already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T553.3 chrF++ · raw 53.3 chrF++ Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,55 into-English + 55 from-English language pairs; source report already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T5 |
| FLORES+ chrF++ (Meta424 directions) | 47.5 chrF++high effort | All 4 recorded results & sources47 chrF++ · raw 47 chrF++ Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,212 into-English + 212 from-English, public devtest, 1,012 sentences; source already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T547.4 chrF++ · raw 47.4 chrF++ Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,212 into-English + 212 from-English, public devtest, 1,012 sentences; source already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T547.5 chrF++ · raw 47.5 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,212 into-English + 212 from-English, public devtest, 1,012 sentences; source already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T547.5 chrF++ · raw 47.5 chrF++ Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,212 into-English + 212 from-English, public devtest, 1,012 sentences; source already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T5 |
| BOUQuET chrF++ (Meta548 directions) | 36.5 chrF++high effort | All 4 recorded results & sources35.6 chrF++ · raw 35.6 chrF++ Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,274 into-English + 274 from-English; source already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T536.3 chrF++ · raw 36.3 chrF++ Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,274 into-English + 274 from-English; source already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T536.5 chrF++ · raw 36.5 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,274 into-English + 274 from-English; source already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T536.5 chrF++ · raw 36.5 chrF++ Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 5. High is launch reference; all efforts preserved. Temperature 1/top_p 0.95/top_k 64. chrF++,274 into-English + 274 from-English; source already uses 0–100 points. https://arxiv.org/html/2610.24628v1#A1.T5 |
| Muse Glimmer quantization · mean accuracy across 15 benchmarks | 75.7%unknown effort | All 5 recorded results & sources75.7% · raw 75.7 % Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: BF16.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T375.6% · raw 75.6 % Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: EvoPress.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T375.3% · raw 75.3 % Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: Greedy-KL.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T374.8% · raw 74.8 % Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: Q4KM.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T374.4% · raw 74.4 % Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: NVFP4/MXFP8.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T3 |
| Muse Glimmer quantization · mean response length across 15 benchmarks | 11070 tokens/responseunknown effort | All 5 recorded results & sources11070 tokens/response · raw 11070 tokens/response Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: BF16.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T311417 tokens/response · raw 11417 tokens/response Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: EvoPress.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T310754 tokens/response · raw 10754 tokens/response Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: Greedy-KL.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T310774 tokens/response · raw 10774 tokens/response Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: Q4KM.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T310988 tokens/response · raw 10988 tokens/response Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization: NVFP4/MXFP8.15-benchmark mix; no individual score inferred. Official/experimental BF16/EvoPress/Greedy-KL/Q4KM/NVFP4-MXFP8 configuration; not different registry models. https://arxiv.org/html/2610.24628v1#S7.T3 |
| OmnilingualGAIA2 · eng_Latn (English) (Meta) | 73.7%high effort | All 4 recorded results & sources73.7% · raw 73.7 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; eng_Latn (English). https://arxiv.org/html/2610.24628v1#A2.T773.7% · raw 73.7 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; eng_Latn (English). https://arxiv.org/html/2610.24628v1#A2.T773.7% · raw 73.7 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; eng_Latn (English). https://arxiv.org/html/2610.24628v1#A2.T773.9% · raw 73.9 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; eng_Latn (English). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · spa_Latn (Spanish) (Meta) | 65%high effort | All 4 recorded results & sources61.6% · raw 61.6 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; spa_Latn (Spanish). https://arxiv.org/html/2610.24628v1#A2.T763.9% · raw 63.9 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; spa_Latn (Spanish). https://arxiv.org/html/2610.24628v1#A2.T765% · raw 65 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; spa_Latn (Spanish). https://arxiv.org/html/2610.24628v1#A2.T764.4% · raw 64.4 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; spa_Latn (Spanish). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · fra_Latn (French) (Meta) | 64%high effort | All 4 recorded results & sources60.5% · raw 60.5 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; fra_Latn (French). https://arxiv.org/html/2610.24628v1#A2.T763.3% · raw 63.3 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; fra_Latn (French). https://arxiv.org/html/2610.24628v1#A2.T764% · raw 64 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; fra_Latn (French). https://arxiv.org/html/2610.24628v1#A2.T764.6% · raw 64.6 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; fra_Latn (French). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · por_Latn (Portuguese) (Meta) | 64.9%high effort | All 4 recorded results & sources62.7% · raw 62.7 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; por_Latn (Portuguese). https://arxiv.org/html/2610.24628v1#A2.T764.6% · raw 64.6 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; por_Latn (Portuguese). https://arxiv.org/html/2610.24628v1#A2.T764.9% · raw 64.9 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; por_Latn (Portuguese). https://arxiv.org/html/2610.24628v1#A2.T765.3% · raw 65.3 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; por_Latn (Portuguese). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · ind_Latn (Indonesian) (Meta) | 62.1%high effort | All 4 recorded results & sources59.2% · raw 59.2 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; ind_Latn (Indonesian). https://arxiv.org/html/2610.24628v1#A2.T762% · raw 62 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; ind_Latn (Indonesian). https://arxiv.org/html/2610.24628v1#A2.T762.1% · raw 62.1 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; ind_Latn (Indonesian). https://arxiv.org/html/2610.24628v1#A2.T762.1% · raw 62.1 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; ind_Latn (Indonesian). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · ita_Latn (Italian) (Meta) | 61.8%high effort | All 4 recorded results & sources58.8% · raw 58.8 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; ita_Latn (Italian). https://arxiv.org/html/2610.24628v1#A2.T760.7% · raw 60.7 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; ita_Latn (Italian). https://arxiv.org/html/2610.24628v1#A2.T761.8% · raw 61.8 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; ita_Latn (Italian). https://arxiv.org/html/2610.24628v1#A2.T762.3% · raw 62.3 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; ita_Latn (Italian). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · tur_Latn (Turkish) (Meta) | 60.1%high effort | All 4 recorded results & sources57.6% · raw 57.6 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; tur_Latn (Turkish). https://arxiv.org/html/2610.24628v1#A2.T760.1% · raw 60.1 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; tur_Latn (Turkish). https://arxiv.org/html/2610.24628v1#A2.T760.1% · raw 60.1 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; tur_Latn (Turkish). https://arxiv.org/html/2610.24628v1#A2.T760.7% · raw 60.7 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; tur_Latn (Turkish). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · cmn_Hans (Chinese) (Meta) | 56.4%high effort | All 4 recorded results & sources55.4% · raw 55.4 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; cmn_Hans (Chinese). https://arxiv.org/html/2610.24628v1#A2.T756.5% · raw 56.5 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; cmn_Hans (Chinese). https://arxiv.org/html/2610.24628v1#A2.T756.4% · raw 56.4 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; cmn_Hans (Chinese). https://arxiv.org/html/2610.24628v1#A2.T756.4% · raw 56.4 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; cmn_Hans (Chinese). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · deu_Latn (German) (Meta) | 65.9%high effort | All 4 recorded results & sources62% · raw 62 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; deu_Latn (German). https://arxiv.org/html/2610.24628v1#A2.T763.2% · raw 63.2 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; deu_Latn (German). https://arxiv.org/html/2610.24628v1#A2.T765.9% · raw 65.9 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; deu_Latn (German). https://arxiv.org/html/2610.24628v1#A2.T765.7% · raw 65.7 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; deu_Latn (German). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · hin_Deva (Hindi) (Meta) | 53.6%high effort | All 4 recorded results & sources56.5% · raw 56.5 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; hin_Deva (Hindi). https://arxiv.org/html/2610.24628v1#A2.T759.7% · raw 59.7 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; hin_Deva (Hindi). https://arxiv.org/html/2610.24628v1#A2.T753.6% · raw 53.6 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; hin_Deva (Hindi). https://arxiv.org/html/2610.24628v1#A2.T754.1% · raw 54.1 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; hin_Deva (Hindi). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · jpn_Jpan (Japanese) (Meta) | 50%high effort | All 4 recorded results & sources49.8% · raw 49.8 % Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; jpn_Jpan (Japanese). https://arxiv.org/html/2610.24628v1#A2.T753% · raw 53 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; jpn_Jpan (Japanese). https://arxiv.org/html/2610.24628v1#A2.T750% · raw 50 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; jpn_Jpan (Japanese). https://arxiv.org/html/2610.24628v1#A2.T749.3% · raw 49.3 % Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; jpn_Jpan (Japanese). https://arxiv.org/html/2610.24628v1#A2.T7 |
| OmnilingualGAIA2 · Δ vs. English (Meta) | -13.3 pphigh effort | All 4 recorded results & sources-15.3 pp · raw -15.3 pp Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; Δ vs. English. https://arxiv.org/html/2610.24628v1#A2.T7-13 pp · raw -13 pp Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; Δ vs. English. https://arxiv.org/html/2610.24628v1#A2.T7-13.3 pp · raw -13.3 pp Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; Δ vs. English. https://arxiv.org/html/2610.24628v1#A2.T7-13.4 pp · raw -13.4 pp Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 7; Δ vs. English. https://arxiv.org/html/2610.24628v1#A2.T7 |
| MMLU-ProX mean tokens per rollout | 2453 tokens/rollouthigh effort | All 4 recorded results & sources557 tokens/rollout · raw 557 tokens/rollout Alternative · low effort · Own vendor Source/record date: 2026-10-21 Meta report Table 8, accuracy versus generation length, Muse Glimmer (low) https://arxiv.org/html/2610.24628v1#A2.T81180 tokens/rollout · raw 1180 tokens/rollout Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report Table 8, accuracy versus generation length, Muse Glimmer (medium) https://arxiv.org/html/2610.24628v1#A2.T82453 tokens/rollout · raw 2453 tokens/rollout Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 8, accuracy versus generation length, Muse Glimmer (high) https://arxiv.org/html/2610.24628v1#A2.T83397 tokens/rollout · raw 3397 tokens/rollout Alternative · xhigh effort · Own vendor Source/record date: 2026-10-21 Meta report Table 8, accuracy versus generation length, Muse Glimmer (xhigh) https://arxiv.org/html/2610.24628v1#A2.T8 |
| WMT24++ X→eng chrF++ (Meta) | 55.7 chrF++high effort | All 1 recorded result & sources55.7 chrF++ · raw 55.7 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 9; 55 directions. High reasoning. https://arxiv.org/html/2610.24628v1#A2.T9 |
| WMT24++ eng→Y chrF++ (Meta) | 51 chrF++high effort | All 1 recorded result & sources51 chrF++ · raw 51 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 9; 55 directions. High reasoning. https://arxiv.org/html/2610.24628v1#A2.T9 |
| FLORES+ X→eng chrF++ (Meta) | 53.7 chrF++high effort | All 1 recorded result & sources53.7 chrF++ · raw 53.7 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 9; 212 directions. High reasoning. https://arxiv.org/html/2610.24628v1#A2.T9 |
| FLORES+ eng→Y chrF++ (Meta) | 41.4 chrF++high effort | All 1 recorded result & sources41.4 chrF++ · raw 41.4 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 9; 212 directions. High reasoning. https://arxiv.org/html/2610.24628v1#A2.T9 |
| BOUQuET X→eng chrF++ (Meta) | 41.4 chrF++high effort | All 1 recorded result & sources41.4 chrF++ · raw 41.4 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 9; 274 directions. High reasoning. https://arxiv.org/html/2610.24628v1#A2.T9 |
| BOUQuET eng→Y chrF++ (Meta) | 31.7 chrF++high effort | All 1 recorded result & sources31.7 chrF++ · raw 31.7 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 9; 274 directions. High reasoning. https://arxiv.org/html/2610.24628v1#A2.T9 |
| Overall X↔eng chrF++ (Meta) | 41.3 chrF++high effort | All 1 recorded result & sources41.3 chrF++ · raw 41.3 chrF++ Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report Table 9; 972 directions. High reasoning. https://arxiv.org/html/2610.24628v1#A2.T9 |
| MBCT (Meta Glimmer) | 41.5%unknown effort | All 1 recorded result & sources41.5% · raw 41.5 % Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card chem/bio table; Effort unreported; capability accuracy, not modeled harm risk. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| HPCT (Meta Glimmer) | 52.3%unknown effort | All 1 recorded result & sources52.3% · raw 52.3 % Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card chem/bio table; Effort unreported; capability accuracy, not modeled harm risk. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| VCT (Meta Glimmer) | 37%unknown effort | All 1 recorded result & sources37% · raw 37 % Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card chem/bio table; Effort unreported; capability accuracy, not modeled harm risk. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| WMDP (Bio) (Meta Glimmer) | 86.5%unknown effort | All 1 recorded result & sources86.5% · raw 86.5 % Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card chem/bio table; Effort unreported; capability accuracy, not modeled harm risk. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| WMDP (Chem) (Meta Glimmer) | 75.2%unknown effort | All 1 recorded result & sources75.2% · raw 75.2 % Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card chem/bio table; Effort unreported; capability accuracy, not modeled harm risk. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| Lab Bench (ProtocolQA) (Meta Glimmer) | 80.2%unknown effort | All 1 recorded result & sources80.2% · raw 80.2 % Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card chem/bio table; Effort unreported; capability accuracy, not modeled harm risk. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| Muse Glimmer Nvidia RTX 5090 · baseline | 74.9 tok/sunknown effort | All 1 recorded result & sources74.9 tok/s · raw 74.9 tok/s Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card local hardware case study; K-Quant-17GB + quantized DFlash; batch 1, greedy decoding; RTX llama.cpp/Apple ExecuTorch. Source-rounded speedup retained; not recomputed. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| Muse Glimmer Nvidia RTX 5090 · DFlash | 233.4 tok/sunknown effort | All 2 recorded results & sources233.4 tok/s · raw 233.4 tok/s Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card local hardware case study; K-Quant-17GB + quantized DFlash; batch 1, greedy decoding; RTX llama.cpp/Apple ExecuTorch. Source-rounded speedup retained; not recomputed. https://huggingface.co/meta-models/Muse-Glimmer-30B233 tok/s · raw 233 tok/s Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Meta launch figure rounded DFlash mean decode speed, K-Quant-17GB + quantized drafter, batch 1 greedy decoding. Exact decimal model-card table remains headline. Unlabeled range whiskers are not estimated. https://lookaside.fbsbx.com/elementpath/media/?media_id=1786825739136571&version=1786319248 |
| Muse Glimmer Nvidia RTX 5090 · speedup | 3.1 ×unknown effort | All 1 recorded result & sources3.1 × · raw 3.1 × Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card local hardware case study; K-Quant-17GB + quantized DFlash; batch 1, greedy decoding; RTX llama.cpp/Apple ExecuTorch. Source-rounded speedup retained; not recomputed. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| Muse Glimmer Apple M4 Max · baseline | 23.7 tok/sunknown effort | All 1 recorded result & sources23.7 tok/s · raw 23.7 tok/s Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card local hardware case study; K-Quant-17GB + quantized DFlash; batch 1, greedy decoding; RTX llama.cpp/Apple ExecuTorch. Source-rounded speedup retained; not recomputed. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| Muse Glimmer Apple M4 Max · DFlash | 37.8 tok/sunknown effort | All 2 recorded results & sources37.8 tok/s · raw 37.8 tok/s Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card local hardware case study; K-Quant-17GB + quantized DFlash; batch 1, greedy decoding; RTX llama.cpp/Apple ExecuTorch. Source-rounded speedup retained; not recomputed. https://huggingface.co/meta-models/Muse-Glimmer-30B38 tok/s · raw 38 tok/s Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Meta launch figure rounded DFlash mean decode speed, K-Quant-17GB + quantized drafter, batch 1 greedy decoding. Exact decimal model-card table remains headline. Unlabeled range whiskers are not estimated. https://lookaside.fbsbx.com/elementpath/media/?media_id=1786825739136571&version=1786319248 |
| Muse Glimmer Apple M4 Max · speedup | 1.5 ×unknown effort | All 1 recorded result & sources1.5 × · raw 1.5 × Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card local hardware case study; K-Quant-17GB + quantized DFlash; batch 1, greedy decoding; RTX llama.cpp/Apple ExecuTorch. Source-rounded speedup retained; not recomputed. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| Muse Glimmer Apple M5 Max · baseline | 26.6 tok/sunknown effort | All 1 recorded result & sources26.6 tok/s · raw 26.6 tok/s Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card local hardware case study; K-Quant-17GB + quantized DFlash; batch 1, greedy decoding; RTX llama.cpp/Apple ExecuTorch. Source-rounded speedup retained; not recomputed. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| Muse Glimmer Apple M5 Max · DFlash | 50.2 tok/sunknown effort | All 2 recorded results & sources50.2 tok/s · raw 50.2 tok/s Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card local hardware case study; K-Quant-17GB + quantized DFlash; batch 1, greedy decoding; RTX llama.cpp/Apple ExecuTorch. Source-rounded speedup retained; not recomputed. https://huggingface.co/meta-models/Muse-Glimmer-30B50 tok/s · raw 50 tok/s Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Meta launch figure rounded DFlash mean decode speed, K-Quant-17GB + quantized drafter, batch 1 greedy decoding. Exact decimal model-card table remains headline. Unlabeled range whiskers are not estimated. https://lookaside.fbsbx.com/elementpath/media/?media_id=1786825739136571&version=1786319248 |
| Muse Glimmer Apple M5 Max · speedup | 1.8 ×unknown effort | All 1 recorded result & sources1.8 × · raw 1.8 × Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Meta model card local hardware case study; K-Quant-17GB + quantized DFlash; batch 1, greedy decoding; RTX llama.cpp/Apple ExecuTorch. Source-rounded speedup retained; not recomputed. https://huggingface.co/meta-models/Muse-Glimmer-30B |
| Muse Glimmer quantization · relative degradation | 0.2%unknown effort | All 2 recorded results & sources0.2% · raw 0.2 % Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization K-Quant-Dynamic; source average degradation over 15 common benchmarks. Default BF16 is not assigned zero (the source dash stays missing). https://huggingface.co/meta-models/Muse-Glimmer-30B1% · raw 1 % Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Quantization K-Quant-17GB; source average degradation over 15 common benchmarks. Default BF16 is not assigned zero (the source dash stays missing). https://huggingface.co/meta-models/Muse-Glimmer-30B |
| Muse Glimmer PTQ · text ΔNLL | 0.0021 ΔNLLunknown effort | All 8 recorded results & sources0.0016 ΔNLL · raw 0.0016 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization Uniform MXFP8. Held-out text NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0021 ΔNLL · raw 0.0021 ΔNLL Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization EvoPress. Held-out text NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0027 ΔNLL · raw 0.0027 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization Greedy-KL. Held-out text NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0069 ΔNLL · raw 0.0069 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization NVFP4/MXFP8. Held-out text NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0076 ΔNLL · raw 0.0076 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization Q4KM. Held-out text NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0146 ΔNLL · raw 0.0146 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization AWQ. Held-out text NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0180 ΔNLL · raw 0.018 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization Uniform NVFP4. Held-out text NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0183 ΔNLL · raw 0.0183 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization GPTQ. Held-out text NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=11 |
| Muse Glimmer PTQ · code ΔNLL | 0.0010 ΔNLLunknown effort | All 8 recorded results & sources0.0010 ΔNLL · raw 0.001 ΔNLL Headline · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization EvoPress. Held-out code NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0012 ΔNLL · raw 0.0012 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization Uniform MXFP8. Held-out code NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0014 ΔNLL · raw 0.0014 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization Greedy-KL. Held-out code NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0036 ΔNLL · raw 0.0036 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization Q4KM. Held-out code NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0036 ΔNLL · raw 0.0036 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization NVFP4/MXFP8. Held-out code NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0082 ΔNLL · raw 0.0082 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization AWQ. Held-out code NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0092 ΔNLL · raw 0.0092 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization Uniform NVFP4. Held-out code NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=110.0109 ΔNLL · raw 0.0109 ΔNLL Alternative · unknown effort · Own vendor Source/record date: 2026-10-21 Figure 3 labeled bar read visually. Quantization GPTQ. Held-out code NLL difference; experimental variant diagnostic, not a separately released model. https://arxiv.org/pdf/2610.24628v1#page=11 |
| Linguini per-question exact match | 17.9%high effort | All 1 recorded result & sources17.9% · raw 17.9 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report section B.6. 894 questions, distinct from whole-problem exact match. https://arxiv.org/html/2610.24628v1#A2.SS6.SSS2 |
| MMLU-ProX excluding Wolof (Meta 28 languages) | 81.5%high effort | All 1 recorded result & sources81.5% · raw 81.5 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report section B.6. Excludes Wolof; separate from full 29-language population. https://arxiv.org/html/2610.24628v1#A2.SS6.SSS2 |
| MMLU-ProX mean English-to-language gap excluding Wolof | 2.6 pphigh effort | All 1 recorded result & sources2.6 pp · raw 2.6 pp Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report section B.6. Mean cross-language performance gap; descriptive spread, not overall accuracy. https://arxiv.org/html/2610.24628v1#A2.SS6.SSS2 |
| OmnilingualGAIA2 search capability (Hindi) | 69.9%high effort | All 2 recorded results & sources69.9% · raw 69.9 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report section B.6. Search capability only; separate from four-capability macro. https://arxiv.org/html/2610.24628v1#A2.SS6.SSS181.8% · raw 81.8 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report section B.6. Search capability only; separate from four-capability macro. https://arxiv.org/html/2610.24628v1#A2.SS6.SSS1 |
| OmnilingualGAIA2 search capability (Japanese) | 68.5%high effort | All 2 recorded results & sources68.5% · raw 68.5 % Headline · high effort · Own vendor Source/record date: 2026-10-21 Meta report section B.6. Search capability only; separate from four-capability macro. https://arxiv.org/html/2610.24628v1#A2.SS6.SSS177.8% · raw 77.8 % Alternative · medium effort · Own vendor Source/record date: 2026-10-21 Meta report section B.6. Search capability only; separate from four-capability macro. https://arxiv.org/html/2610.24628v1#A2.SS6.SSS1 |
Independent evaluators
Evaluator harnesses are distinct from vendor measurements. Missing coverage is not a failed test.
| Benchmark / evaluator | Headline score | Evidence |
|---|---|---|
| Intelligence Index · Artificial Analysis | 17high effort | All 1 recorded result & sources17 · raw 17 index Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Version4.3.2 ten-component index, not task accuracy. https://artificialanalysis.ai/models/muse-glimmer |
| Output speed · Artificial Analysis | 116.3 tok/shigh effort | All 1 recorded result & sources116.3 tok/s · raw 116.3 tok/s Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Median provider output speed reported in summary, distinct from OpenRouter live endpoints. https://artificialanalysis.ai/models/muse-glimmer |
| Cost per Intelligence Index task · Artificial Analysis | $0.06/taskhigh effort | All 1 recorded result & sources$0.06/task · raw 0.06 USD/task Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Weighted workload-specific API-equivalent cost, not a token tariff. https://artificialanalysis.ai/models/muse-glimmer |
| Output tokens from Intelligence Index (millions) · Artificial Analysis | 59 M tokenshigh effort | All 1 recorded result & sources59 M tokens · raw 59 M tokens Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Source-rounded complete evaluation output-token quantity. https://artificialanalysis.ai/models/muse-glimmer |
| AA-Briefcase v1.1 Elo · Artificial Analysis | 477high effort | All 1 recorded result & sources477 · raw 477 Elo Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Native Elo; do not invert the normalized rounded zero-percent chart. https://artificialanalysis.ai/models/muse-glimmer |
| AA-Omniscience Index · Artificial Analysis | -33high effort | All 1 recorded result & sources-33 · raw -33 index Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| AA-Omniscience accuracy · Artificial Analysis | 27%high effort | All 1 recorded result & sources27% · raw 27 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| AA-Omniscience non-hallucination rate · Artificial Analysis | 18%high effort | All 1 recorded result & sources18% · raw 18 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Published non-hallucination rate (1 minus hallucination rate), higher is better. Separate from hallucination rate. https://artificialanalysis.ai/models/muse-glimmer |
| AutomationBench (AA) · Artificial Analysis | 7%high effort | All 1 recorded result & sources7% · raw 7 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| Humanity's Last Exam (AA harness) · Artificial Analysis | 22%high effort | All 1 recorded result & sources22% · raw 22 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| GDP.PDF (AA) · Artificial Analysis | 10%high effort | All 1 recorded result & sources10% · raw 10 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| Terminal-Bench 4.0 (AA) · Artificial Analysis | 1%high effort | All 1 recorded result & sources1% · raw 1 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| AA-LCR v1.1 (AA) · Artificial Analysis | 83%high effort | All 1 recorded result & sources83% · raw 83 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| SciCode (AA) · Artificial Analysis | 45%high effort | All 1 recorded result & sources45% · raw 45 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Source currently flags this result under review. https://artificialanalysis.ai/models/muse-glimmer |
| CritPt (AA) · Artificial Analysis | 3%high effort | All 1 recorded result & sources3% · raw 3 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Source currently flags this result under review. https://artificialanalysis.ai/models/muse-glimmer |
| Terminal-Bench Science 0.1 (AA) · Artificial Analysis | 0%high effort | All 1 recorded result & sources0% · raw 0 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| FrontierCode 1.1 Main (AA harness) · Weighted rubric score · Artificial Analysis | 14%high effort | All 1 recorded result & sources14% · raw 14 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. FrontierCode is a separate evaluation, not an Intelligence Index component. https://artificialanalysis.ai/models/muse-glimmer |
| AA-AnalystAgent · Artificial Analysis | 20%high effort | All 1 recorded result & sources20% · raw 20 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| EnterpriseOps-Gym-AA · Artificial Analysis | 35%high effort | All 1 recorded result & sources35% · raw 35 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| MMMU-Pro (AA) · Artificial Analysis | 74%high effort | All 1 recorded result & sources74% · raw 74 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| Terminal-Bench 2.1 (AA) · Artificial Analysis | 52%high effort | All 1 recorded result & sources52% · raw 52 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| GPQA Diamond (AA) · Artificial Analysis | 84%high effort | All 1 recorded result & sources84% · raw 84 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| τ³-Banking (AA) · Artificial Analysis | 24%high effort | All 1 recorded result & sources24% · raw 24 % Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. https://artificialanalysis.ai/models/muse-glimmer |
| AA Industry Index Oct2026 · Finance & Accounting · Artificial Analysis | 18high effort | All 1 recorded result & sources18 · raw 18 index Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Current published industry aggregate; no computed average. Components include versioned GDPval2.1/Briefcase1.1 where specified; separate from historical unversioned indices. https://artificialanalysis.ai/models/muse-glimmer |
| AA Industry Index Oct2026 · Strategy & Ops · Artificial Analysis | 15high effort | All 1 recorded result & sources15 · raw 15 index Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Current published industry aggregate; no computed average. Components include versioned GDPval2.1/Briefcase1.1 where specified; separate from historical unversioned indices. https://artificialanalysis.ai/models/muse-glimmer |
| AA Industry Index Oct2026 · Legal · Artificial Analysis | 18high effort | All 1 recorded result & sources18 · raw 18 index Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Current published industry aggregate; no computed average. Components include versioned GDPval2.1/Briefcase1.1 where specified; separate from historical unversioned indices. https://artificialanalysis.ai/models/muse-glimmer |
| AA Industry Index Oct2026 · Healthcare & Medical · Artificial Analysis | 18high effort | All 1 recorded result & sources18 · raw 18 index Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Current published industry aggregate; no computed average. Components include versioned GDPval2.1/Briefcase1.1 where specified; separate from historical unversioned indices. https://artificialanalysis.ai/models/muse-glimmer |
| AA Industry Index Oct2026 · Engineering · Artificial Analysis | 20high effort | All 1 recorded result & sources20 · raw 20 index Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Current published industry aggregate; no computed average. Components include versioned GDPval2.1/Briefcase1.1 where specified; separate from historical unversioned indices. https://artificialanalysis.ai/models/muse-glimmer |
| AA Industry Index Oct2026 · Economics · Artificial Analysis | 25high effort | All 1 recorded result & sources25 · raw 25 index Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Current published industry aggregate; no computed average. Components include versioned GDPval2.1/Briefcase1.1 where specified; separate from historical unversioned indices. https://artificialanalysis.ai/models/muse-glimmer |
| AA Openness Index Oct2026 · Artificial Analysis | 44high effort | All 1 recorded result & sources44 · raw 44 index Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Published openness assessment on 0–100 scale; not model quality or task accuracy. https://artificialanalysis.ai/models/muse-glimmer |
| Time to first token (AA) · Artificial Analysis | 1.26 shigh effort | All 1 recorded result & sources1.26 s · raw 1.26 s Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. FAQ native median across serving providers; excludes reasoning-to-first-answer distinction. Not network-free model computation. https://artificialanalysis.ai/models/muse-glimmer |
| Cost to run Intelligence Index · Artificial Analysis | $146.00high effort | All 1 recorded result & sources$146.00 · raw 146 USD Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Complete Intelligence Index workload cost; explicit highlighted total, not a stacked component or token tariff. https://artificialanalysis.ai/models/muse-glimmer |
| Output tokens per Intelligence Index task · Artificial Analysis | 19000 tokens/taskhigh effort | All 1 recorded result & sources19000 tokens/task · raw 19000 tokens/task Headline · high effort · Independent evaluator Source/record date: 2026-10-21 Manual public rendered chart/summary observation on 21 October 2026. Exact Muse Glimmer (high) page links the official meta-models/Muse-Glimmer-30B weights. No API, internal app state, scheduled import, hidden precision or inferred run date. Published rounded chart labels retained. Serving checkpoint SHA unpublished. Source rounded19k weighted output tokens/task, including reasoning. No more precision inferred. https://artificialanalysis.ai/models/muse-glimmer |
| AA-PersonalityBench Oct2026 · Index · Expressiveness percentile · Artificial Analysis | 20 percentilehigh effort | All 1 recorded result & sources20 percentile · raw 20 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Rich, varied, well-crafted language. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 20. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Lexically Varied percentile · Artificial Analysis | 29 percentilehigh effort | All 1 recorded result & sources29 percentile · raw 29 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Lexically Varied writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 29. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Eloquent percentile · Artificial Analysis | 32 percentilehigh effort | All 1 recorded result & sources32 percentile · raw 32 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Eloquent writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 32. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Literate percentile · Artificial Analysis | 12 percentilehigh effort | All 1 recorded result & sources12 percentile · raw 12 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Literate writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 12. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Natural Sounding percentile · Artificial Analysis | 43 percentilehigh effort | All 1 recorded result & sources43 percentile · raw 43 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Natural Sounding writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 43. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Metaphorical percentile · Artificial Analysis | 58 percentilehigh effort | All 1 recorded result & sources58 percentile · raw 58 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Metaphorical writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 58. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Index · Organization percentile · Artificial Analysis | 12 percentilehigh effort | All 1 recorded result & sources12 percentile · raw 12 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Well-structured, scannable, answer-first. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 12. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Structured percentile · Artificial Analysis | 35 percentilehigh effort | All 1 recorded result & sources35 percentile · raw 35 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Structured writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 35. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Scannable percentile · Artificial Analysis | 15 percentilehigh effort | All 1 recorded result & sources15 percentile · raw 15 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Scannable writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 15. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Bottom-Line Summary percentile · Artificial Analysis | 9 percentilehigh effort | All 1 recorded result & sources9 percentile · raw 9 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Bottom-Line Summary writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 9. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Answer First percentile · Artificial Analysis | 35 percentilehigh effort | All 1 recorded result & sources35 percentile · raw 35 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Answer First writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 35. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Index · Informativeness percentile · Artificial Analysis | 51 percentilehigh effort | All 1 recorded result & sources51 percentile · raw 51 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Practical depth, evidence, actionable content. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 51. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Information Dense percentile · Artificial Analysis | 49 percentilehigh effort | All 1 recorded result & sources49 percentile · raw 49 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Information Dense writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 49. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Evidence-Based percentile · Artificial Analysis | 35 percentilehigh effort | All 1 recorded result & sources35 percentile · raw 35 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Evidence-Based writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 35. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Resource-Specific percentile · Artificial Analysis | 52 percentilehigh effort | All 1 recorded result & sources52 percentile · raw 52 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Resource-Specific writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 52. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Actionable Guidance percentile · Artificial Analysis | 52 percentilehigh effort | All 1 recorded result & sources52 percentile · raw 52 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Actionable Guidance writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 52. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Pragmatic percentile · Artificial Analysis | 29 percentilehigh effort | All 1 recorded result & sources29 percentile · raw 29 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Pragmatic writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 29. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Index · Transparency percentile · Artificial Analysis | 24 percentilehigh effort | All 1 recorded result & sources24 percentile · raw 24 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Truthful, acknowledges uncertainty, non-sycophantic. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 24. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Truthful percentile · Artificial Analysis | 12 percentilehigh effort | All 1 recorded result & sources12 percentile · raw 12 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Truthful writing trait, judged pairwise among responses to the same questions. A judged writing/personality trait in this question population, not a general factual-accuracy benchmark. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 12. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Admits Uncertainty percentile · Artificial Analysis | 24 percentilehigh effort | All 1 recorded result & sources24 percentile · raw 24 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Admits Uncertainty writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 24. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Disclaims Assumptions percentile · Artificial Analysis | 63 percentilehigh effort | All 1 recorded result & sources63 percentile · raw 63 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Disclaims Assumptions writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 63. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Flags Nuances percentile · Artificial Analysis | 35 percentilehigh effort | All 1 recorded result & sources35 percentile · raw 35 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Flags Nuances writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 35. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Sycophantic percentile · Artificial Analysis | 77 percentilehigh effort | All 1 recorded result & sources77 percentile · raw 77 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Sycophantic writing trait, judged pairwise among responses to the same questions. Native trait direction: higher means more sycophancy; only its contribution to Transparency is inverted. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 77. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Index · Warmth percentile · Artificial Analysis | 62 percentilehigh effort | All 1 recorded result & sources62 percentile · raw 62 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Friendly, humorous, sincere, emotionally attuned. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 62. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Friendly percentile · Artificial Analysis | 92 percentilehigh effort | All 1 recorded result & sources92 percentile · raw 92 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Friendly writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 92. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Humorous percentile · Artificial Analysis | 29 percentilehigh effort | All 1 recorded result & sources29 percentile · raw 29 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Humorous writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 29. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Sincere percentile · Artificial Analysis | 32 percentilehigh effort | All 1 recorded result & sources32 percentile · raw 32 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Sincere writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 32. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Emotionally Intelligent percentile · Artificial Analysis | 77 percentilehigh effort | All 1 recorded result & sources77 percentile · raw 77 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Emotionally Intelligent writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 77. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Index · Conviction percentile · Artificial Analysis | 9 percentilehigh effort | All 1 recorded result & sources9 percentile · raw 9 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Direct, confident positions and constructive pushback. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 9. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Direct percentile · Artificial Analysis | 12 percentilehigh effort | All 1 recorded result & sources12 percentile · raw 12 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Direct writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 12. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Confident percentile · Artificial Analysis | 9 percentilehigh effort | All 1 recorded result & sources9 percentile · raw 9 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Confident writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 9. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Opinionated percentile · Artificial Analysis | 18 percentilehigh effort | All 1 recorded result & sources18 percentile · raw 18 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Opinionated writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 18. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Constructive Pushback percentile · Artificial Analysis | 29 percentilehigh effort | All 1 recorded result & sources29 percentile · raw 29 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Constructive Pushback writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 29. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Index · Imagination percentile · Artificial Analysis | 81 percentilehigh effort | All 1 recorded result & sources81 percentile · raw 81 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Creative, exploratory, open-ended thinking. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 81. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Creative percentile · Artificial Analysis | 55 percentilehigh effort | All 1 recorded result & sources55 percentile · raw 55 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Creative writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 55. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Open-Ended percentile · Artificial Analysis | 92 percentilehigh effort | All 1 recorded result & sources92 percentile · raw 92 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Open-Ended writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 92. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Index · Slop percentile · Artificial Analysis | 73 percentilehigh effort | All 1 recorded result & sources73 percentile · raw 73 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Use of filler and generic language. Source index is the mean of member-trait Elo z-scores followed by percentile ranking across the October2026 evaluated pool. Sycophantic is inverted only when contributing to Transparency; this does not invert the separately reported Sycophantic trait. A higher number means more of this named communication pattern, not better model quality. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 73. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Trait · Slop percentile · Artificial Analysis | 72 percentilehigh effort | All 1 recorded result & sources72 percentile · raw 72 percentile Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Prominence of the Slop writing trait, judged pairwise among responses to the same questions. Blind pairwise LLM judging by Gemini3.8Flash(high), ClaudeSonnet5.5(medium), GLM5.3Flash(max), one randomly assigned judge per comparison with balanced assignments and randomized answer order; reasoning text hidden. Firth-penalized Bradley–Terry trait ratings; ties count as half-win. GPT-6.1Sol(max) anchors Elo1000. Published values here are rounded 1–100 pool-relative percentiles, NOT Elo, raw accuracy or a percentage of tasks passed. The current capture contains 36 judged configurations even though UI selection labels say38of38; the two extra communication-only models have no captured judged values. Underlying Elo and 95%CI values were not captured, and are not inferred from chart widths/colors; methodology uses1000bootstrap refits. The pool-relative scale can move when the evaluated pool changes without a new model execution. Source label: 72. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Characters · Artificial Analysis | 3210 characters/responsehigh effort | All 1 recorded result & sources3210 characters/response · raw 3210 characters/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Characters: Characters in the prose, spaces included, after code and URLs are removed. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 3,210. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Words · Artificial Analysis | 516 words/responsehigh effort | All 1 recorded result & sources516 words/response · raw 516 words/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Words: Tokens that contain a letter or digit; each Chinese, Japanese or Korean character counts as one word (but all questions are in English). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 516. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Sentences · Artificial Analysis | 39.00 sentences/responsehigh effort | All 1 recorded result & sources39.00 sentences/response · raw 39 sentences/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Sentences: Sentences found by a rule-based splitter that handles abbreviations such as “Dr.”. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 39.0. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Paragraphs · Artificial Analysis | 12.20 paragraphs/responsehigh effort | All 1 recorded result & sources12.20 paragraphs/response · raw 12.2 paragraphs/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Paragraphs: Paragraphs of prose; headings, code and the items of a single-spaced list do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 12.2. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Avg. sentence length · Artificial Analysis | 14.10 words/sentencehigh effort | All 1 recorded result & sources14.10 words/sentence · raw 14.1 words/sentence Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Avg. sentence length: Words divided by sentences. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 14.1. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Lists · Artificial Analysis | 2.38 lists/responsehigh effort | All 1 recorded result & sources2.38 lists/response · raw 2.38 lists/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Lists: Top-level bulleted or numbered lists; nested sub-lists do not count separately. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 2.38. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Headings · Artificial Analysis | 1.29 headings/responsehigh effort | All 1 recorded result & sources1.29 headings/response · raw 1.29 headings/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Headings: Headings of any level. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 1.29. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Tables · Artificial Analysis | 0.01 tables/responsehigh effort | All 1 recorded result & sources0.01 tables/response · raw 0.01 tables/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Tables: Tables. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.01. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Em dashes · Artificial Analysis | 0.80 em dashes/responsehigh effort | All 1 recorded result & sources0.80 em dashes/response · raw 0.8 em dashes/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Em dashes: The em dash character (—); double hyphens do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.80. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Contractions · Artificial Analysis | 9.40 contractions/responsehigh effort | All 1 recorded result & sources9.40 contractions/response · raw 9.4 contractions/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Contractions: Contractions such as “don’t”, “you’re” and “it’s”; possessives such as “John’s” do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 9.40. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Exclamations · Artificial Analysis | 0.06 exclamations/responsehigh effort | All 1 recorded result & sources0.06 exclamations/response · raw 0.06 exclamations/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Exclamations: Exclamation marks. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.06. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Questions · Artificial Analysis | 3.74 question marks/responsehigh effort | All 1 recorded result & sources3.74 question marks/response · raw 3.74 question marks/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Questions: Question marks. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 3.74. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Parentheses · Artificial Analysis | 0.07 parenthesis pairs/responsehigh effort | All 1 recorded result & sources0.07 parenthesis pairs/response · raw 0.07 parenthesis pairs/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Parentheses: Closed pairs of parentheses, nested pairs included; unmatched parentheses do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.07. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Quotes · Artificial Analysis | 7.84 quotation marks/responsehigh effort | All 1 recorded result & sources7.84 quotation marks/response · raw 7.84 quotation marks/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Quotes: Quotation marks of any style, such as straight, curly, « » and „ “; apostrophes do not count. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 7.84. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Colons · Artificial Analysis | 7.33 colons/responsehigh effort | All 1 recorded result & sources7.33 colons/response · raw 7.33 colons/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Colons: Colons. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 7.33. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Ellipses · Artificial Analysis | 0.16 ellipses/responsehigh effort | All 1 recorded result & sources0.16 ellipses/response · raw 0.16 ellipses/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Ellipses: Three periods (...) or the ellipsis character (…). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.16. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Bold text · Artificial Analysis | 7.77 bold spans/responsehigh effort | All 1 recorded result & sources7.77 bold spans/response · raw 7.77 bold spans/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Bold text: Uses of bold: each unbroken run of bold text counts once, so a single bolded word and a fully bolded sentence both count as 1. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 7.77. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Italics · Artificial Analysis | 1.39 italic spans/responsehigh effort | All 1 recorded result & sources1.39 italic spans/response · raw 1.39 italic spans/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Italics: Uses of italics: each unbroken run of italic text counts once, so a single italicized word and a fully italicized sentence both count as 1. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 1.39. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Emojis · Artificial Analysis | 0.00 emojis/responsehigh effort | All 1 recorded result & sources0.00 emojis/response · raw 0 emojis/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Emojis: Each emoji as the reader sees it. An emoji built from several characters, such as a family (👨👩👧), still counts as one emoji. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 0.00. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Acronyms · Artificial Analysis | 2.97 acronyms/responsehigh effort | All 1 recorded result & sources2.97 acronyms/response · raw 2.97 acronyms/response Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Acronyms: Two or more capital letters in a row (AI, USA) or dotted forms (U.S.). Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 2.97. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| AA-PersonalityBench Oct2026 · Communication · Ends with a question · Artificial Analysis | 28%high effort | All 1 recorded result & sources28% · raw 28 % Headline · high effort · Independent evaluator Source/record date: 2026-10-19 Public rendered chart labels manually captured by the root agent, then transcribed once from those captures. Published 19 October 2026; source checked 20 October 2026; execution dates unpublished. No system prompt: one response to each of 900 English questions, 30 questions per each of 30 traits. This describes default communication style and trait prominence, not objective answer quality or accuracy. No AA API, internal app-state access, automated collector, paid export or inference was used. Programmatically measured Ends with a question: Whether the answer’s last character is a question mark, reported as the share of responses that do. Source-native average over responses to the same900questions; heatmap colors use pool-relative percentiles, but the visible cell numbers here are native quantities, not those color percentiles. The Characters definition explicitly removes code and URLs; exact feature-specific handling follows the methodology. Average sentence length is words per sentence, not per response. Ends with a question is the share of responses (%); bold/italic count spans, not words or characters. Descriptive communication style, not a quality ranking. Source label: 28%. Exact Muse Glimmer(high) newly approved 21 October; root verified AA model-page official Muse-Glimmer-30B reference link that day. These are historical20October chart observations, not a fresh21October pool capture. No transfer from Spark/Wonder or auxiliary drafter. https://artificialanalysis.ai/evaluations/aa-personalitybench |
| GDPval-AA v2.1 Elo · Artificial Analysis | 787high effort | All 1 recorded result & sources787 · raw 787 Elo Headline · high effort · Independent evaluator Source/record date: 2026-10-20 Root manually captured full public GDPval-AA v2.1 table on20October2026; exact Muse Glimmer(High), Meta, source-native Elo787, displayedCI−24/+24. Source model release month Aug2026 is not execution date. Newly approved21October after exact AA model-page official Muse-Glimmer-30B identity verification. This dated nativeElo is not the normalized model-chart score and is not a fresh21October pool capture. Distinct older benchmark versions remain unchanged. Highest and only captured effort for this exact approved model is headline; no serving precision, hidden effort budget or task cost inferred. https://artificialanalysis.ai/evaluations/gdpval-aa |
Read how we select and source scores or the comparison guide.