Examenos

54.9% · raw 54.9 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Terminus 2 XML; internet enabled; retrieval-obtained solutions count as unsuccessful. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: BF16.

53.7% · raw 53.7 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Terminus 2 XML; internet enabled; retrieval-obtained solutions count as unsuccessful. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: NVFP4.

89.7% · raw 89.7 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: mini-swe-agent; internet disabled. Pro v2 is distinct from older Pro revisions. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: BF16.

88.5% · raw 88.5 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: mini-swe-agent; internet disabled. Pro v2 is distinct from older Pro revisions. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: NVFP4.

74.2% · raw 74.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: mini-swe-agent; internet disabled. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: BF16.

74.1% · raw 74.1 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: mini-swe-agent; internet disabled. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: NVFP4.

80.3% · raw 80.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Single-turn; exact task/date slice not restated. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: BF16.

80.8% · raw 80.8 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Single-turn; exact task/date slice not restated. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: NVFP4.

89.2% · raw 89.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Single-turn; strict/loose scoring variant not restated. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: BF16.

88.1% · raw 88.1 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Single-turn; strict/loose scoring variant not restated. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: NVFP4.

95.1% · raw 95.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Single-turn; prompt/instruction and strict/loose aggregation not restated. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: BF16.

94.6% · raw 94.6 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Single-turn; prompt/instruction and strict/loose aggregation not restated. Recursive release protocol: temperature 1.0, top-p 0.95, top-k 20. Agentic average pass@1 over at least five attempts, up to five-hour timeout; single-turn tests average pass@1 over five attempts. Evaluation date and task counts unpublished. BF16 is the consistent headline; NVFP4 is retained separately, not selected opportunistically per test. Weight precision: NVFP4.

131.2 tok/s · raw 131.22 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Decode speed at 1K input context. Prompt prefill excluded. Vendor hardware/runtime observation, not hosted API speed or end-to-end latency. M4 Pro 48 GiB / MLX. Quantization: local low-bit; exact weight format not restated. Output length and repetitions not restated in this figure.

68.9 tok/s · raw 68.89 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Normalized area under the decode-throughput curve through 128K input context; Muse comparator ends at 124K. Prompt prefill excluded. Vendor hardware/runtime observation, not hosted API speed or end-to-end latency. M4 Pro 48 GiB / MLX. Quantization: local low-bit; exact weight format not restated. Output length and repetitions not restated in this figure.

88 tok/s · raw 87.98 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Decode speed at 1K input context. Prompt prefill excluded. Vendor hardware/runtime observation, not hosted API speed or end-to-end latency. M4 Pro 48 GiB / llama.cpp. Quantization: GGUF; exact quantization not restated. Output length and repetitions not restated in this figure.

47.6 tok/s · raw 47.58 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Normalized area under the decode-throughput curve through 128K input context; Muse comparator ends at 124K. Prompt prefill excluded. Vendor hardware/runtime observation, not hosted API speed or end-to-end latency. M4 Pro 48 GiB / llama.cpp. Quantization: GGUF; exact quantization not restated. Output length and repetitions not restated in this figure.

155.8 tok/s · raw 155.82 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Decode speed at 1K input context. Prompt prefill excluded. Vendor hardware/runtime observation, not hosted API speed or end-to-end latency. RTX Spark N1X / NVFP4. Quantization: NVFP4. Output length and repetitions not restated in this figure.

102 tok/s · raw 101.95 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Normalized area under the decode-throughput curve through 128K input context; Muse comparator ends at 124K. Prompt prefill excluded. Vendor hardware/runtime observation, not hosted API speed or end-to-end latency. RTX Spark N1X / NVFP4. Quantization: NVFP4. Output length and repetitions not restated in this figure.

96.6 tok/s · raw 96.61 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Decode speed at 1K input context. Prompt prefill excluded. Vendor hardware/runtime observation, not hosted API speed or end-to-end latency. RTX Spark N1X / llama.cpp. Quantization: GGUF; exact quantization not restated. Output length and repetitions not restated in this figure.

59.4 tok/s · raw 59.35 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Normalized area under the decode-throughput curve through 128K input context; Muse comparator ends at 124K. Prompt prefill excluded. Vendor hardware/runtime observation, not hosted API speed or end-to-end latency. RTX Spark N1X / llama.cpp. Quantization: GGUF; exact quantization not restated. Output length and repetitions not restated in this figure.

131.2 tok/s · raw 131.2 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Official rec engine figure: Apple M4 Pro 48 GiB, Flint 4-bit weights, 1K input context, 1024 output tokens, five repeats, prefill excluded. Runtime rec. Exact endpoint value is explicitly published in the accessible SVG description; intermediate coordinates are not estimated. Distinct from the blog MLX/AUC protocol and from BF16 or API speed.

22.6 tok/s · raw 22.6 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Official rec engine figure: Apple M4 Pro 48 GiB, Flint 4-bit weights, 252K input context, 1024 output tokens, five repeats, prefill excluded. Runtime rec. Exact endpoint value is explicitly published in the accessible SVG description; intermediate coordinates are not estimated. Distinct from the blog MLX/AUC protocol and from BF16 or API speed.

88 tok/s · raw 88 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Official rec engine figure: Apple M4 Pro 48 GiB, Flint 4-bit weights, 1K input context, 1024 output tokens, five repeats, prefill excluded. Runtime llama-cpp. Exact endpoint value is explicitly published in the accessible SVG description; intermediate coordinates are not estimated. Distinct from the blog MLX/AUC protocol and from BF16 or API speed.

8.6 tok/s · raw 8.6 tok/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-21

Harness: Official rec engine figure: Apple M4 Pro 48 GiB, Flint 4-bit weights, 252K input context, 1024 output tokens, five repeats, prefill excluded. Runtime llama-cpp. Exact endpoint value is explicitly published in the accessible SVG description; intermediate coordinates are not estimated. Distinct from the blog MLX/AUC protocol and from BF16 or API speed.