Examenos

84% · raw 84 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-05

Harness: Reflection launch evaluation; full agent, revision and token-budget settings not published. As reported by Reflection; peer comparison, not an independent evaluator record.

84% · raw 84 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Vendor-run AA-LCR, not directly verified independent AA; revision unpublished. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

81.3% · raw 81.3 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. No v1.1 revision or exact context budget is established by this unversioned vendor label. As reported by KRAFTON for this comparison peer; does not override the peer’s own vendor headline.

83.3% · raw 83.3 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-25

Vendor-run AA-LCR, not directly verified independent AA; revision unpublished. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

81.3% · raw 81.3 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-03

Harness: Aleph Alpha eval-framework/Harbor, released post-trained checkpoint. Kolibri reasoning effort high; peers use vendor-documented sampling. Technical report Tables 27-29 (pp. 96-101). Vendor-run AA-LCR, avg@3; not an independent AA measurement. As reported by Aleph Alpha for Qwen3.8 27B; preserves existing own-vendor headline when present.

80.3% · raw 80.3 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. No v1.1 revision or exact context budget is established by this unversioned vendor label. As reported by KRAFTON for this comparison peer; does not override the peer’s own vendor headline.

82% · raw 82 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-25

Vendor-run AA-LCR, not directly verified independent AA; revision unpublished. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

81% · raw 81 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Sage-A1.5 report Table 1. Final released checkpoint. 100 multi-document questions, reference LLM judge, accuracy averaged 3 runs; source version not explicit.

79.7% · raw 79.7 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-05

Harness: Reflection launch evaluation; full agent, revision and token-budget settings not published. As reported by Reflection; peer comparison, not an independent evaluator record.

79.7% · raw 79.7 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-21

Sage-A1.5 report Table 1. As reported by Shanghai AI Laboratory; mixed own-report/best-effort reproduction sources; exact peer effort unreported. 100 multi-document questions, reference LLM judge, accuracy averaged 3 runs; source version not explicit. Non-headline addition 2026-10-21: preserves the existing own-vendor or earlier configured headline; this source remains reachable as distinct vendor evidence.

78.2% · raw 78.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-17

Harness: Reflection final instruction-tuned BF16; temp=0.7, top_p=0.99. Effort and run date unpublished. Earlier preview results are retained separately.

79.4% · raw 79.4 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-17

Harness: Reflection final instruction-tuned FP8; temp=0.7, top_p=0.99. Effort and run date unpublished. Earlier preview results are retained separately.

78.7% · raw 78.7 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-17

Harness: Reflection final instruction-tuned NVFP4; temp=0.7, top_p=0.99. Effort and run date unpublished. Earlier preview results are retained separately.

79.3% · raw 79.3 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-05

Harness: Reflection launch evaluation; full agent, revision and token-budget settings not published. Beam early-access preview, reported by Reflection. Demoted 2026-10-17: retained preview observation; released BF16 card now supplies the current headline.

74.7% · raw 74.7 %

Headline · max effort · Own vendor

Source/record date: 2026-07-16

AA-LCR; Moonshot cites Artificial Analysis as of 2026-07-23 Provenance corrected 2026-10-06: published on Moonshot’s own model card; vendor quotes AA, not a peer-vendor comparison.

88.7% · raw 88.7 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-05

Harness: Reflection launch evaluation; full agent, revision and token-budget settings not published. As reported by Reflection; peer comparison, not an independent evaluator record.

88.7% · raw 88.7 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-21

Sage-A1.5 report Table 1. As reported by Shanghai AI Laboratory; mixed own-report/best-effort reproduction sources; exact peer effort unreported. 100 multi-document questions, reference LLM judge, accuracy averaged 3 runs; source version not explicit. Non-headline addition 2026-10-21: preserves the existing own-vendor or earlier configured headline; this source remains reachable as distinct vendor evidence.

72% · raw 72 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-28

From the DiffuSpace launch blog general-benchmarks.jpg table (same values as the agentic-benchmarks.jpg chart and the 28 October X post): "AA-LCR". Vendor EvalScope run, not an Artificial Analysis harness observation; revision not stated. EvalScope; DiffuSpace reproduced every baseline under the same settings with each model's recommended decoding. Thinking mode; effort label not published.

72% · raw 72 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-25

Vendor-run AA-LCR, not directly verified independent AA; revision unpublished. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

71% · raw 71 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-14

As reported by NVIDIA in the BF16 baseline row, explicitly linked to official zai-org/GLM-5.3-Flash-BF16, the same Z.ai post-trained model in a published BF16 precision variant. This is the baseline, not NVIDIA NVFP4 quantized scores. Published raw 0–1 value 0.71 converted to 71.0%. Harness: NVIDIA baseline evaluation; temperature=1.0, top_p=0.95; max_new_tokens=327,680. Exact evaluated-model effort, task sampling, grader, scaffold revision and run date are unpublished. Own-vendor headline wins when present. AA-named suites remain vendor/peer-harness evidence, separate from independent AA entries. Source checked 14 October 2026.

71% · raw 71 %

Headline · unknown effort · Own vendor

Source/record date: 2026-08-11

Harness: Upstage vendor-cited Artificial Analysis August 2026 observation, not direct current AA data. The chart labels these internally measured while prose cites AA; provenance disagreement is retained and not resolved by guessing. Exact approved Solar Pro 4; Solar Open 2 is a separate, unapproved comparison peer. Checked 16 October 2026.

71% · raw 71 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-08-20

Later vendor press release repeats August AA-LCR observation; retained as a source alternative to launch article.

68.3% · raw 68.3 %

Headline · high effort · Own vendor

Source/record date: 2026-10-03

Harness: Aleph Alpha eval-framework/Harbor, released post-trained checkpoint. Kolibri reasoning effort high; peers use vendor-documented sampling. Technical report Tables 27-29 (pp. 96-101). Vendor-run AA-LCR, avg@3; not an independent AA measurement.

68% · raw 68 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-08

from chart/figure on Inception Mercury 2.5 release post (alt: Mercury 2.5 vs Mercury 2.0)

70% · raw 70 %

Alternative · medium effort · Peer vendor

Source/record date: 2026-10-28

Harness: DiffuSpace reproduction for this peer, general-benchmarks.jpg table (same values as the agentic-benchmarks.jpg chart and the 28 October X post): "AA-LCR". Vendor EvalScope run, not an Artificial Analysis harness observation; revision not stated. EvalScope; DiffuSpace reproduced every baseline under the same settings with each model's recommended decoding. Mercury 2.5 Thinking Medium per the table note.

67.3% · raw 67.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. No v1.1 revision or exact context budget is established by this unversioned vendor label.

57.6% · raw 57.56 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-19

Harness: NVIDIA same-suite vendor comparison of exact Gemma4 26B-A4B-it (confirmed figure label). As reported by NVIDIA; effort and inference precision unpublished for the peer.

67% · raw 67 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. No v1.1 revision or exact context budget is established by this unversioned vendor label. As reported by KRAFTON for this comparison peer; does not override the peer’s own vendor headline.

52% · raw 52 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: NVIDIA release NeMo Gym/Evaluator; BF16 reference weights. AA-LCR. Source comparison metrics are vendor-run, not independent AA. Thinking enabled without an API effort label; benchmark-specific sampling/repeats are in the official recipes.