Examenos

65.7% · raw 65.7 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: K3.1 Pro uses Claude Code; peers are publisher-selected maxima from their releases or the official board, not a synchronized common-harness rerun. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

38.1% · raw 38.1 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: K3.1 Pro uses Claude Code. Footnote 2 explicitly attributes the displayed DeepSeek 15.7 to the older September model, not 1017; the separately disclosed internal 1017 result is 9.3. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

51.7% · raw 51.7 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: K3.1 Pro and DeepSeek 1017 use Claude Code; remaining peers are official leaderboard observations. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

72.8% · raw 72.8 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Open models: random 1,000 cases from 17,900 Vision2Code, max, single-shot official prompts/scoring. Peer proprietary values are cited. Voxel IoU on a published 0–100 scale; not full-set or an assumed common five-run subset. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

92.7% · raw 92.7 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Open models: random 1,000 cases from 17,900 Vision2Code, max, Claude Code; proprietary values are system-card/report citations. Voxel IoU on a published 0–100 scale; tool and no-tool protocols remain separate. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

31.5% · raw 31.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Binary threshold: original task score ≥95% becomes 1, otherwise 0. K3.1 Pro uses Claude Code; DeepSeek/GLM are release citations and other peers Vals citations. Underlying peer source quantities/harnesses are heterogeneous; no uniform fully-resolved protocol is inferred. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

75.5% · raw 75.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: K3.1 Pro uses Claude Code; peer observations are cited from their own technical reports. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

36.5 score · raw 36.5 score

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: K3.1 Pro/DeepSeek 1017: Claude Code on H20 rather than H100, DeepSeek 1017 max judges. Peers are official board values with different hardware/judges. Native performance points; not an inferred task-success percentage or v1.1 rerun. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

77.1% · raw 77.1 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Internal coding suite; K3.1 Pro uses Claude Code; the footnote states max effort for other tested models and possible proprietary fallback behavior. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

61.20% · raw 61.2 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Internal multi-step engineering domains. K3.1 Pro uses Claude Code; other models max, with possible proprietary fallbacks; population and run dates unpublished. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

70% · raw 70 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Full 2,500-question evaluation with tools; do not confuse with text-only HLE or no-tools HLE. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

92.1% · raw 92.1 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Context compaction at 200K, maximum 500 steps and forced answer at the limit; temperature 1.0/top-p 1.0. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

80.6% · raw 80.6 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Official v11 dataset; Kimi K3 judge. K3.1 Pro uses Kimi Code, DeepSeek/GLM Claude Code, Astra Codex; Claude scores are system-card citations. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

93.4% · raw 93.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Python-tool variant, four-run mean. Judge not specified; distinct from plain CharXiv and MiniMax judge-specific cells. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

90% · raw 90 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Python tools; four-run mean. Original input order preserved and images prepended to text. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

85.7% · raw 85.7 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Original PDFs supplied to IPython sandbox; Gemini 3.5 Flash medium judge with official prompt. Five runs; task-macro average of fraction of rubric criteria passed. This is partial rubric reward, not strict all-pass GDP.PDF. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

90.2% · raw 90.2 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Python tools; five-run mean; distinct from no-tools ChartoGraphy. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

95.4% · raw 95.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Python tools; four-run mean. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

63% · raw 63 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Python tools on the main split; explicitly pass@5, not mean per-attempt accuracy. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

43.2% · raw 43.2 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Official v1.1 environment-aligned 122-task population; online reference-answer/task-requirement bypasses zeroed. Kimi uses Kimi Code, DeepSeek Claude Code; peers official board. Not the independent ALE v1 Full/Unlicensed quantities. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

44.2% · raw 44.2 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: 86 CLI tasks within the 122-task v1.1 population; online reference-answer/task-requirement bypasses zeroed. Kimi Code/Claude Code; remaining peers board citations. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

73.3% · raw 73.3 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Vendor-cited AA observation; exact reward aggregation is not restated. This stays separate from directly verified independent AA scores and Meta strict-all-assertion scores. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

1846 · raw 1846 Elo

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Vendor-cited dated AA native Elo; pool-dependent and not a normalized accuracy percentage. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

1678 · raw 1678 Elo

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Vendor-cited AA native Elo; not rubric pass rate or another Briefcase revision. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

67.2% · raw 67.2 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Vendor cites Mercor Apex Agents board; exact run date and individual source configuration are unpublished. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

64.4% · raw 64.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Entire PDF corpus rendered as images; no machine-readable text; external reference-answer retrieval zeroed. Astra Codex, others Claude Code. Preserve v2 separately from earlier Pro. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

67.4% · raw 67.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Entire PDF corpus rendered as images; no machine-readable text; external reference-answer retrieval zeroed. Astra Codex, others Claude Code. Not Pro v2. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

58.6% · raw 58.6 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Official finance-agent v2 harness; vendor-run result, distinct from independent Vals Finance Agent and Vals Finance Index. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

23.7% · raw 23.7 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Public 1,251-task population, AA Stirrup harness; all-rubric all-pass. Vendor-run, not independent Harvey LAB-AA v1.1 on 120 tasks. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

44.6% · raw 44.6 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Early open-source v1 task population, official OpenHands harness. Do not infer later benchmark revision or task count. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

79.6% · raw 79.6 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Official OSWorld v2 repo, Kimi Agent harness; Astra is a Meta Wonder report citation. Task count and partial/binary aggregation not restated, so this observation is nonrankable and does not enter the explicit 108-task cells. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

37.8 score · raw 37.8 score

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Long-horizon management/decision simulation. Native score, no percentage scale or task population specified. Table 37.8 differs from figure 37.0; both retained. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

37 score · raw 37 score

Alternative · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Finance figure 37.0 conflicts with final table 37.8. Table precision/explicit value is headline; both preserved without averaging. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

1456 · raw 1456 Elo

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Presentation generation across python-pptx,pptxjs,Kimi DSL, integrated search/image generation. Automated grader calibrated on 10,000+ human judgments. Native pool-dependent Elo, not Arena Text. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

957 · raw 957 Elo

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Direct/reference-image few-shot graphic design across python-pptx,pptxjs,Kimi DSL; search/image generation and automated grader calibrated on 10,000+ human judgments. Native source pool Elo. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

8.3% · raw 8.3 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Internal challenging de-identified practical-work prompts, fact-checking agents. Percentage of factual claims with errors; lower better. Stress-test population, not everyday error rate. Final release 8.3 supersedes draft-doc 6.85; protocols not proven identical. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

34.6% · raw 34.6 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: No Python. PDF page images resized to longest edge 1,152 pixels, interleaved with Docling text; Gemini 3.5 Flash medium judge. Strict all-rubric pass rate among fully scored responses; five runs. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

51.1% · raw 51.1 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: No tools, full 2,500 questions; starred peers explicitly use a text-only subset and are routed separately. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

39.4% · raw 39.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: No tools, full 1,000 questions; starred peers explicitly use a text-only subset and are routed separately. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

94.2% · raw 94.2 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Vendor-reported GPQA Diamond; exact execution date and serving weight hash unpublished. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

89% · raw 89 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: No Python, CharXiv reasoning questions; four-run mean. Do not borrow MiniMax GPT-5.5 or Meta gpt-oss judge details. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

84.1% · raw 84.1 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: No tools, official protocol, original input order preserved and images prepended; four-run mean. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

29.8% · raw 29.8 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Mean of 5 no-tool attempts; no Python tools; five runs. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

39% · raw 39 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Pass@5 no-tool attempts; no Python tools; five runs. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

68.3% · raw 68.3 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Vendor-reported multimodal perception; four-run mean; exact suite revision not restated. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

5.4% · raw 5.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Internal challenging de-identified practical-work prompts, fact-checking agents. Percentage of factual claims with errors; lower better. Distinct from the tool-enabled Hard population. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

74.7% · raw 74.7 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Official harness, 300 turns, best of 5 seeds per environment, fraction of 16 capability flags; arbitrary code execution counts as all 16. Not an ordinary pass@1 task-success percentage. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

20.7% · raw 20.7 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: K3.1 Pro uses Kimi Code, 6h wall-clock/task; peers vendor/system-card citations at 6h. Native reported percent; not exploit counts or 2h protocol. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

81% · raw 81 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Expert-curated source-based vulnerability tasks with hints/binaries/mitigations, random subset of runs, no hard time/output-token cap. Kimi Code versus peer Claude Code. Full table/footnote 3 identify DeepSeek 1017; figure shortens its name. Unreleased no-risk-reduction ablation is excluded. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

57.6 score · raw 57.6 score

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Formula linkage, financial logic and final workbook quality; internal real-world workflows; native source points. Figure-only result visually verified. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

70.4 score · raw 70.4 score

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Auditable research under evidence and cutoff constraints; native source points. Figure-only result visually verified. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

42.4% · raw 42.4 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Responses containing at least one factual error, expressed as percent of K3’s rate. 42.4 is relative, not 42.4% of responses; underlying absolute rates unpublished. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

11.2 s · raw 11.2 s

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: One internal simulator/controller construction case: 167 missions, 420 reviews, 36 shoulder moment-arm corrections. Not a standardized benchmark or general model-quality ranking. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

64.5% · raw 64.5 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Internal xdev figure, visually verified. Kimi uses Kimi Code; DeepSeek/GLM Claude Code. Full table and footnote 3 identify DeepSeek 1017 despite the shortened figure label. No hard time/output-token caps; no assumption that pass@3 equals average success. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

159800000 tokens/successful-trajectory · raw 159800000 tokens/successful-trajectory

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Internal xdev figure, visually verified. Kimi uses Kimi Code; DeepSeek/GLM Claude Code. Full table and footnote 3 identify DeepSeek 1017 despite the shortened figure label. No hard time/output-token caps; no assumption that pass@3 equals average success. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

546000 tokens/successful-trajectory · raw 546000 tokens/successful-trajectory

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: Internal xdev figure, visually verified. Kimi uses Kimi Code; DeepSeek/GLM Claude Code. Full table and footnote 3 identify DeepSeek 1017 despite the shortened figure label. No hard time/output-token caps; no assumption that pass@3 equals average success. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

55 · raw 55 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

73.5 tok/s · raw 73.5 tok/s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

$2.11/task · raw 2.11 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

150 M tokens · raw 150 M tokens

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

1678 · raw 1678 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

51% · raw 51 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

56% · raw 56 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

73% · raw 73 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

1846 · raw 1846 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

52% · raw 52 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

31% · raw 31 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

67% · raw 67 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

86% · raw 86 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

61% · raw 61 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate. Publisher marks SciCode under review.

25% · raw 25 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.

84% · raw 84 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-23

The model page calls this the reasoning variant but does not label its exact effort. Do not infer max from Moonshot API defaults. Percent values and Elo are source-rounded chart labels. Speed73.5 is the explicit summary value, distinct from chart-rounded73. The unverified secondary55.1 claim is not used. SciCode is under review. No inference precision or checkpoint hash is published. Manual public rendered label review, 23 October 2026. This quantity is independent evaluator evidence; vendor-cited AA values remain separate.