Examenos

70% · raw 70 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Full 2,500-question evaluation with tools; do not confuse with text-only HLE or no-tools HLE. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

67.7% · raw 67.7 %

Headline · max effort · Own vendor

Source/record date: 2026-09-22

Humanity's Last Exam with tools

57.4% · raw 57.4 %

Alternative · low effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

63% · raw 63 %

Alternative · medium effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

63.9% · raw 63.9 %

Alternative · high effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

66.4% · raw 66.4 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

67.7% · raw 67.7 %

Alternative · max effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

67.7% · raw 67.7 %

Alternative · max effort · Peer vendor

Source/record date: 2026-10-23

Harness: Full 2,500-question evaluation with tools; do not confuse with text-only HLE or no-tools HLE. Table labels max with fallback; chart notes mention possible Claude fallbacks/GPT cyberguards. This is not a guaranteed single-model-only execution. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. As reported by Moonshot; preserve the peer’s own-vendor headline and source-specific setup.

65% · raw 65 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-09-03

As reported by OpenAI | Demoted 2026-09-25: duplicate of headline from Anthropic Claude Opus 5.5 announce (anthropic.com/claude-opus-5-5); model vendor's own (Anthropic) figure preferred over peer-vendor OpenAI comparison table (rule a); Anthropic source is also later-dated (2026-09-22). Values differ (65.0 here vs 65.6 Anthropic); not reconciled. source_type relabeled official→vendor-comparison (OpenAI reporting an Anthropic model).

65.6% · raw 65.6 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-09-22

As reported by Anthropic Opus 5.5 announce

65% · raw 65 %

Alternative · max effort · Peer vendor

Source/record date: 2026-09-20

Harness: StepFun vendor-reported evaluation; Text-only subset, explicitly distinguished by source footnote; not full HLE with tools. Full agent/grader/task-subset settings unpublished unless stated. Source inspected 8 October 2026; evaluation date unknown unless stated. Approved peer, max effort as labelled; not an independent run. Existing headline retained; alternate vendor configuration/evidence, no averaging.

65.6% · raw 65.6 %

Alternative · max effort · Peer vendor

Source/record date: 2026-10-23

Harness: Full 2,500-question evaluation with tools; do not confuse with text-only HLE or no-tools HLE. Table labels max with fallback; chart notes mention possible Claude fallbacks/GPT cyberguards. This is not a guaranteed single-model-only execution. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. As reported by Moonshot; preserve the peer’s own-vendor headline and source-specific setup.

64.5% · raw 64.5 %

Headline · max effort · Own vendor

Source/record date: 2026-09-28

HLE with tools at max

47.6% · raw 47.6 %

Alternative · low effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

50.5% · raw 50.5 %

Alternative · medium effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

56.7% · raw 56.7 %

Alternative · high effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

62% · raw 62 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

64.5% · raw 64.5 %

Alternative · max effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

63.9% · raw 63.9 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-10

HLE w/ tools Pass@1

63.9% · raw 63.9 %

Alternative · max effort · Peer vendor

Source/record date: 2026-10-05

From chart/figure: Reflection Beam Reasoning table, visually read in full resolution 14 October 2026. HLE with tools, not the separate no-tools row. Exact subset, tools/harness, budget, grader and run date are unpublished. As reported by Reflection; preserve the peer own-vendor headline. The DeepSeek column explicitly says DeepSeek V4.1 Flash Max; GLM effort is unspecified. Figure header lists Beam 1M context and Apache 2.0, but these do not establish current serving limit or released weights; keep current model metadata sourced from API docs/HF availability.

62.5% · raw 62.5 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-09-10

As reported in DeepSeek-V4.1-Flash HF comparison table (GLM-5.3 column)

62.5% · raw 62.5 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-05

From chart/figure: Reflection Beam Reasoning table, visually read in full resolution 14 October 2026. HLE with tools, not the separate no-tools row. Exact subset, tools/harness, budget, grader and run date are unpublished. As reported by Reflection; preserve the peer own-vendor headline. The DeepSeek column explicitly says DeepSeek V4.1 Flash Max; GLM effort is unspecified. Figure header lists Beam 1M context and Apache 2.0, but these do not establish current serving limit or released weights; keep current model metadata sourced from API docs/HF availability.

62.5% · raw 62.5 %

Alternative · max effort · Peer vendor

Source/record date: 2026-10-15

Harness: Muse internal bash/browser tools with HF/Kaggle/PapersWithCode/LMSYS blocked; question subset unpublished. Peers from self-reports; not proof of identical tool or task budgets. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

62.5% · raw 62.5 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-21

Sage-A1.5 report Table 1. As reported by Shanghai AI Laboratory; mixed own-report/best-effort reproduction sources; exact peer effort unreported. Academic multimodal questions with tools; exact split/version unreported. Non-headline addition 2026-10-21: preserves the existing own-vendor or earlier configured headline; this source remains reachable as distinct vendor evidence.

62.5% · raw 62.5 %

Alternative · max effort · Peer vendor

Source/record date: 2026-10-23

Harness: Full 2,500-question evaluation with tools; do not confuse with text-only HLE or no-tools HLE. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. As reported by Moonshot; preserve the peer’s own-vendor headline and source-specific setup.

61% · raw 61 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-10-15

Harness: Muse internal bash/browser tools with HF/Kaggle/PapersWithCode/LMSYS blocked; question subset unpublished. Peers from self-reports; not proof of identical tool or task budgets. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Launch xhigh (strength 512), temperature/top_p=1.0, top_k disabled. Source checked 15 October 2026; exact execution date unpublished.

58.8% · raw 58.8 %

Headline · max effort · Peer vendor

Source/record date: 2026-10-23

Harness: Full 2,500-question evaluation with tools; do not confuse with text-only HLE or no-tools HLE. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. As reported by Moonshot; preserve the peer’s own-vendor headline and source-specific setup.

38.6% · raw 38.6 %

Alternative · low effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs.

45% · raw 45 %

Alternative · medium effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

50.1% · raw 50.1 %

Alternative · high effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

54.8% · raw 54.8 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs. Alternative evidence; existing headline/configuration retained. No averaging.

57.4% · raw 57.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-07

Anthropic Haiku 5.5 system card, p. 119. Web search/fetch and code execution; contamination-source restrictions; adaptive effort matrix, five trials. Web-search fees excluded from chart costs.

57.2% · raw 57.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-03

57.2% · raw 57.2 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-09-22

As reported by Anthropic Opus 5.5 announce | Demoted 2026-09-25: duplicate of headline from OpenAI GPT-6 Astra announce (openai.com/index/gpt-6-astra); vendor official preferred over Anthropic peer comparison table (rule a); same value.

57.2% · raw 57.2 %

Alternative · max effort · Peer vendor

Source/record date: 2026-09-20

Harness: StepFun vendor-reported evaluation; Text-only subset, explicitly distinguished by source footnote; not full HLE with tools. Full agent/grader/task-subset settings unpublished unless stated. Source inspected 8 October 2026; evaluation date unknown unless stated. Approved peer, max effort as labelled; not an independent run. Existing headline retained; alternate vendor configuration/evidence, no averaging.

57.2% · raw 57.2 %

Alternative · max effort · Peer vendor

Source/record date: 2026-10-23

Harness: Full 2,500-question evaluation with tools; do not confuse with text-only HLE or no-tools HLE. Table labels max with fallback; chart notes mention possible Claude fallbacks/GPT cyberguards. This is not a guaranteed single-model-only execution. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. As reported by Moonshot; preserve the peer’s own-vendor headline and source-specific setup.

56.2% · raw 56.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-08-03

Alibaba’s Qwen3.8-Max table.

56% · raw 56 %

Headline · max effort · Own vendor

Source/record date: 2026-07-16

HLE-Full with general tools

56% · raw 56 %

Alternative · max effort · Peer vendor

Source/record date: 2026-09-20

Harness: StepFun vendor-reported evaluation; Text-only subset, explicitly distinguished by source footnote; not full HLE with tools. Full agent/grader/task-subset settings unpublished unless stated. Source inspected 8 October 2026; evaluation date unknown unless stated. Approved peer, max effort as labelled; not an independent run. Existing headline retained; alternate vendor configuration/evidence, no averaging.

56% · raw 56 %

Alternative · max effort · Peer vendor

Source/record date: 2026-10-15

Harness: Muse internal bash/browser tools with HF/Kaggle/PapersWithCode/LMSYS blocked; question subset unpublished. Peers from self-reports; not proof of identical tool or task budgets. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

56% · raw 56 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-21

Sage-A1.5 report Table 1. As reported by Shanghai AI Laboratory; mixed own-report/best-effort reproduction sources; exact peer effort unreported. Academic multimodal questions with tools; exact split/version unreported. Non-headline addition 2026-10-21: preserves the existing own-vendor or earlier configured headline; this source remains reachable as distinct vendor evidence.

55.3% · raw 55.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-08-26

55.1% · raw 55.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Sage-A1.5 report Table 1. Final released checkpoint. Academic multimodal questions with tools; exact split/version unreported.

51.1% · raw 51.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-17

Harness: Reflection final instruction-tuned BF16; temp=0.7, top_p=0.99. Effort and run date unpublished. HLE text modality only; source does not identify a newer HLE-Diamond subset. Earlier preview results are retained separately.

51.1% · raw 51.1 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-17

Harness: Reflection final instruction-tuned FP8; temp=0.7, top_p=0.99. Effort and run date unpublished. HLE text modality only; source does not identify a newer HLE-Diamond subset. Earlier preview results are retained separately.

51.1% · raw 51.1 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-17

Harness: Reflection final instruction-tuned NVFP4; temp=0.7, top_p=0.99. Effort and run date unpublished. HLE text modality only; source does not identify a newer HLE-Diamond subset. Earlier preview results are retained separately.

51.2% · raw 51.2 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-05

From chart/figure: Reflection Beam Reasoning table, visually read in full resolution 14 October 2026. HLE with tools, not the separate no-tools row. Exact subset, tools/harness, budget, grader and run date are unpublished. Beam effort is unspecified. Figure header lists Beam 1M context and Apache 2.0, but these do not establish current serving limit or released weights; keep current model metadata sourced from API docs/HF availability. Demoted 2026-10-17: retained preview observation; released BF16 card now supplies the current headline.

51.2% · raw 51.2 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-15

Harness: Muse internal bash/browser tools with HF/Kaggle/PapersWithCode/LMSYS blocked; question subset unpublished. Peers from self-reports; not proof of identical tool or task budgets. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

36.9% · raw 36.9 %

Headline · high effort · Own vendor

Source/record date: 2026-10-20

Harness: SparkLLM own run at thinking high per its card, temperature 1.0, top_p 0.95; exact label “HLE(w tools)” in the card figure. Self-reported; run date unpublished.

26.5% · raw 26.5 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Search tool enabled. Report dated19June2026, not execution date. Visually checked numeric table.

19.9% · raw 19.9 %

Headline · high effort · Own vendor

Source/record date: 2026-10-19

Harness: SparkLLM own run at thinking high per its card, temperature 1.0, top_p 0.95; exact label “HLE(w tools)” in the card figure. Self-reported; run date unpublished.

19% · raw 19 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-20

SparkLLM-measured peer in its Spark-X2.6 card figure, label “HLE(w tools)”, not a Google result. Starred peer values on that figure are copied from other cards and are not stored.

17.2% · raw 17.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Google final instruction-tuned report Table5; thinking enabled, no effort tier inferred. Search tool enabled. Report dated19June2026, not execution date. Visually checked numeric table.

10.1% · raw 10.1 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-20

SparkLLM-measured peer in its Spark-X2.6 card figure, label “HLE(w tools)”, not a Google result. Starred peer values on that figure are copied from other cards and are not stored.

6.6% · raw 6.6 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-19

SparkLLM-measured peer in its Spark-X2.6 card figure, label “HLE(w tools)”, not a Google result. Starred peer values on that figure are copied from other cards and are not stored.