Examenos

87.7% · raw 87.7 %

Headline · max effort · Peer vendor

Source/record date: 2026-10-23

Harness: No tools, official protocol, original input order preserved and images prepended; four-run mean. Table labels max with fallback; chart notes mention possible Claude fallbacks/GPT cyberguards. This is not a guaranteed single-model-only execution. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. As reported by Moonshot; preserve the peer’s own-vendor headline and source-specific setup.

87% · raw 87 %

Headline · max effort · Peer vendor

Source/record date: 2026-09-20

Harness: StepFun vendor-reported evaluation; Vendor score 76.0, not independent AA 76.36. Full agent/grader/task-subset settings unpublished unless stated. Source inspected 8 October 2026; evaluation date unknown unless stated. Approved peer, max effort as labelled; not an independent run.

86.9% · raw 86.9 %

Alternative · max effort · Peer vendor

Source/record date: 2026-10-23

Harness: No tools, official protocol, original input order preserved and images prepended; four-run mean. Table labels max with fallback; chart notes mention possible Claude fallbacks/GPT cyberguards. This is not a guaranteed single-model-only execution. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. As reported by Moonshot; preserve the peer’s own-vendor headline and source-specific setup.

86.3% · raw 86.3 %

Headline · max effort · Peer vendor

Source/record date: 2026-10-23

Harness: No tools, official protocol, original input order preserved and images prepended; four-run mean. Table labels max with fallback; chart notes mention possible Claude fallbacks/GPT cyberguards. This is not a guaranteed single-model-only execution. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. As reported by Moonshot; preserve the peer’s own-vendor headline and source-specific setup.

84.1% · raw 84.1 %

Headline · max effort · Own vendor

Source/record date: 2026-10-23

Harness: No tools, official protocol, original input order preserved and images prepended; four-run mean. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. K3.1 Pro defaults: max, temperature 1.0, top-p 0.95 unless otherwise stated.

82.7% · raw 82.7 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-09-23

As reported from Alibaba Qwen3.8-Max-0902 published comparison table (DataCamp transcription); MMMU-Pro

81.6% · raw 81.6 %

Headline · max effort · Own vendor

Source/record date: 2026-07-16

MMMU-Pro without tools (with tools 83.4 stored separately)

81% · raw 81 %

Alternative · max effort · Peer vendor

Source/record date: 2026-09-20

Harness: StepFun vendor-reported evaluation; Vendor score 76.0, not independent AA 76.36. Full agent/grader/task-subset settings unpublished unless stated. Source inspected 8 October 2026; evaluation date unknown unless stated. Approved peer, max effort as labelled; not an independent run. Existing headline retained; alternate vendor configuration/evidence, no averaging.

80.7% · raw 80.7 %

Headline · unknown effort · Own vendor

Source/record date: 2026-07-09

no tools

80.6% · raw 80.6 %

Headline · max effort · Peer vendor

Source/record date: 2026-10-23

Harness: No tools, official protocol, original input order preserved and images prepended; four-run mean. Published 23 October 2026; checked 23 October 2026. Actual run dates, serving checkpoint hashes and complete compute budgets are unpublished. Vendor and independently verified results remain separate. As reported by Moonshot; preserve the peer’s own-vendor headline and source-specific setup.

79% · raw 79 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-10

MMMU-Pro; from inclusionAI card chart

76.9% · raw 76.9 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native %.

76.9% · raw 76.88 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-14

As reported by NVIDIA in the BF16 baseline row, explicitly linked to official zai-org/GLM-5.3-Flash-BF16, the same Z.ai post-trained model in a published BF16 precision variant. This is the baseline, not NVIDIA NVFP4 quantized scores. Published raw 0–1 value 0.7688 converted to 76.88%. Harness: NVIDIA baseline evaluation; temperature=1.0, top_p=0.95; max_new_tokens=327,680. Exact evaluated-model effort, task sampling, grader, scaffold revision and run date are unpublished. Own-vendor headline wins when present. AA-named suites remain vendor/peer-harness evidence, separate from independent AA entries. Source checked 14 October 2026.

76% · raw 76 %

Headline · high effort · Own vendor

Source/record date: 2026-09-20

Harness: StepFun vendor-reported evaluation; Vendor score 76.0, not independent AA 76.36. Full agent/grader/task-subset settings unpublished unless stated. Source inspected 8 October 2026; evaluation date unknown unless stated. Step 5 Preview high effort.

73.8% · raw 73.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native %.

69.7% · raw 69.74 %

Headline · none effort · Peer vendor

Source/record date: 2026-10-12

Harness: As reported by OpenBMB for exact Qwen3.8 27B, Instruct mode, fixed 4x visual compression. Not Qwen3.6 27B, nor a thinking run. Task subsets/judge/budgets unpublished. Checked 13 October 2026; evaluation date unpublished.

69.7% · raw 69.71 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

67.7% · raw 67.66 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

69.1% · raw 69.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native %.

52.6% · raw 52.6 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native %.

44.2% · raw 44.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-19

Harness: Google report Table6; thinking,1120 visual tokens; execution date and exact output budget unpublished. Native %.