Examenos

91.7% · raw 91.72 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

90.9% · raw 90.87 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

80.7% · raw 80.67 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

80.1% · raw 80.13 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

75.5% · raw 75.47 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

75.1% · raw 75.12 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

54.1% · raw 54.14 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

51.4% · raw 51.41 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

80.2% · raw 80.22 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

80% · raw 80 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

88% · raw 88 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

87.3% · raw 87.3 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

69.7% · raw 69.71 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

67.7% · raw 67.66 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

86.5% · raw 86.45 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

84.8% · raw 84.8 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

74.2% · raw 74.24 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

72.9% · raw 72.93 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

91.2% · raw 91.2 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. OCRBench normalized 0–100 score, not raw 0–1000 points. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

90.4% · raw 90.4 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. OCRBench normalized 0–100 score, not raw 0–1000 points. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

68.5% · raw 68.47 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

65.7% · raw 65.67 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

94.2% · raw 94.24 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

93.3% · raw 93.25 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

75.6% · raw 75.6 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

69.2% · raw 69.2 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

81.1% · raw 81.14 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

78.3% · raw 78.29 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

85% · raw 84.99 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

81.7% · raw 81.72 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

70.3% · raw 70.33 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

68.1% · raw 68.12 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

16% · raw 16 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

15.2% · raw 15.2 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

64.4% · raw 64.42 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

63.8% · raw 63.83 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

4.7 score · raw 4.72 score

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

4.7 score · raw 4.67 score

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

21.9% · raw 21.88 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

22.9% · raw 22.92 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

90.9% · raw 90.93 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

89.6% · raw 89.61 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

87% · raw 86.99 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

85% · raw 85.01 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

65.4% · raw 65.41 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

64.6% · raw 64.62 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

76.4% · raw 76.4 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

74.7% · raw 74.7 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

68.9% · raw 68.9 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

67.6% · raw 67.6 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

68.1% · raw 68.12 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 4x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

68% · raw 68.02 %

Alternative · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public-report comparison using official checkpoints, Instruct mode (thinking disabled), 16x visual token compression. Exact task subsets, judge and decoding budgets unpublished. Headline is the better published compression configuration; alternative remains available. Checked 13 October 2026; evaluation date unpublished.

90.2% · raw 90.21 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public reports using official checkpoints, text-only Instruct mode (thinking disabled). Exact task split, prompts and output budgets unpublished. Checked 13 October 2026; evaluation date unpublished.

83.5% · raw 83.45 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public reports using official checkpoints, text-only Instruct mode (thinking disabled). Exact task split, prompts and output budgets unpublished. Checked 13 October 2026; evaluation date unpublished.

61.2% · raw 61.23 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public reports using official checkpoints, text-only Instruct mode (thinking disabled). Exact task split, prompts and output budgets unpublished. Checked 13 October 2026; evaluation date unpublished.

47.7% · raw 47.67 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public reports using official checkpoints, text-only Instruct mode (thinking disabled). Exact task split, prompts and output budgets unpublished. Checked 13 October 2026; evaluation date unpublished.

76.5% · raw 76.46 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public reports using official checkpoints, text-only Instruct mode (thinking disabled). Exact task split, prompts and output budgets unpublished. AIME26@avg16: 16-run average; not pass@16. Checked 13 October 2026; evaluation date unpublished.

56.6% · raw 56.57 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public reports using official checkpoints, text-only Instruct mode (thinking disabled). Exact task split, prompts and output budgets unpublished. Checked 13 October 2026; evaluation date unpublished.

60.1% · raw 60.06 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public reports using official checkpoints, text-only Instruct mode (thinking disabled). Exact task split, prompts and output budgets unpublished. BFCL-V4 overall as labelled by vendor; per-component scores not supplied. Checked 13 October 2026; evaluation date unpublished.

84.7% · raw 84.65 %

Headline · none effort · Own vendor

Source/record date: 2026-10-12

Harness: OpenBMB local evaluation/public reports using official checkpoints, text-only Instruct mode (thinking disabled). Exact task split, prompts and output budgets unpublished. Checked 13 October 2026; evaluation date unpublished.

3 tok/s · raw 2.97 tok/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: Unoptimized SSD baseline on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

9.8 tok/s · raw 9.81 tok/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: LES scheduling and loading on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

14.9 tok/s · raw 14.92 tok/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: LES plus mixed-precision quantization on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

17.4 tok/s · raw 17.39 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: Full LES + DSpark on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

1 x · raw 1 x

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: Unoptimized SSD baseline on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

3.3 x · raw 3.3 x

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: LES scheduling and loading on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

5 x · raw 5.02 x

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: LES plus mixed-precision quantization on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

5.9 x · raw 5.86 x

Headline · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: Full LES + DSpark on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

3.3 x · raw 3.3 x

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: LES scheduling and loading on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

1.5 x · raw 1.52 x

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: LES plus mixed-precision quantization on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.

1.2 x · raw 1.17 x

Headline · unknown effort · Own vendor

Source/record date: 2026-10-12

Harness: Full LES + DSpark on iPad Pro 11-inch M4 (16GB), SSD expert loading. Test data: MMMU, MathVista, CharXiv_reasoning_val, OCRBench, HumanEval, MBPP, AIME2025/2026, Math500, internal dialogue Arena. Decode speed = generated tokens / total decode time. Quantized device configuration, not BF16 cloud-provider throughput. Full LES+DSpark is headline; stage alternatives retained. Checked 13 October 2026; evaluation date unpublished. Device-test reasoning mode is not specified; do not infer it from the separate Instruct scorecard.