Examenos

80.9% · raw 80.9 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Terminus 2; benchmark revision 2.1. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

23.8% · raw 23.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

mini-swe-agent v2.4.6. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

65.5% · raw 65.5 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

mini-swe-agent v2.4.6; dataset revision unspecified. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

60.3% · raw 60.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness/revision unpublished. Footnote marker alone does not establish mini-swe-agent. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

50.8% · raw 50.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

1,304 task-specific rubrics; GPT-6 Sol medium; mini-swe-agent v2.4.6. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

53.0% · raw 53 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Claude Code. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

52.6 score · raw 52.6 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

SciCode aggregation unspecified. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

61.1% · raw 61.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Version unspecified; do not infer v1.1. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

54.2% · raw 54.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

v1.0.6, 600 public tasks. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

44.8 score · raw 44.8 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Claude Code plus search; 35 occupations, aggregate weighting unpublished. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

84.8 score · raw 84.8 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Revised rubrics; not evaluator Elo. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

78.3% · raw 78.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

General subset, 157 single-turn tasks, Pass³. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

70.4% · raw 70.4 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

No per-request max_tokens limit. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

63.3 score · raw 63.3 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Official repository; exact population and aggregation unspecified. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

41.0% · raw 41 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Official repository; do not assume all 1,150 tasks were run. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

$1025437.00 · raw 1025437 USD

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Only the last 10 user-dialogue turns retained. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

48.8 score · raw 48.8 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Domain, revision and aggregation unspecified. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

89.5% · raw 89.5 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

GPQA Diamond. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

95.2% · raw 95.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Sampling and aggregation unpublished. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

84% · raw 84 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Vendor-run AA-LCR, not directly verified independent AA; revision unpublished. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

19.7 score · raw 19.7 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

38.1 score · raw 38.1 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

46.1 score · raw 46.1 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

72.0 score · raw 72 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

41.9 score · raw 41.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

18.2 score · raw 18.2 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

43.9 score · raw 43.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

16.0 score · raw 16 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

39.9 score · raw 39.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

39.9 score · raw 39.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

10.3 score · raw 10.3 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

33.6 score · raw 33.6 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

54.1 score · raw 54.1 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

27.8 score · raw 27.8 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

43.6 score · raw 43.6 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

43.9 score · raw 43.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

50.0 score · raw 50 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

23.7 score · raw 23.7 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

55.6 score · raw 55.6 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

57.4 score · raw 57.4 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

37.5 score · raw 37.5 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

21.6 score · raw 21.6 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

68.5 score · raw 68.5 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

32.5 score · raw 32.5 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

21.1 score · raw 21.1 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

59.0 score · raw 59 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

65.3 score · raw 65.3 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

47.5 score · raw 47.5 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

62.5 score · raw 62.5 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

63.9 score · raw 63.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

72.1 score · raw 72.1 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

33.3 score · raw 33.3 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

72.2 score · raw 72.2 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

28.8 score · raw 28.8 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.

57.3 score · raw 57.3 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Mean task score within occupation, 0–100 points; not pass rate. Nanbeige publisher evaluation; highest thinking effort, exact native effort/budgets, run dates, precision and peer checkpoint hashes unpublished. Nanbeige preserve_thinking=true. Recommended chat/agent generation settings are not verified executed benchmark settings.