Examenos

76.5% · raw 76.5 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown.

3.5 · raw 3.5 index

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. Signed source index, not accuracy or hallucination rate. Public/private population and exact index revision unpublished. Both figure themes were visually checked; table value matches the labeled figure.

84.3% · raw 84.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. Both figure themes were visually checked; table value matches the labeled figure.

93.3% · raw 93.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown.

84.1% · raw 84.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. February 2026 competition; sampling/aggregation unpublished.

84.2% · raw 84.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. Both figure themes were visually checked; table value matches the labeled figure.

59% · raw 59 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown.

81.6% · raw 81.6 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. Version 6 explicitly stated; source does not restate its problem-date window or aggregation.

38.7% · raw 38.7 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown.

70.8% · raw 70.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: mini-SWE-agent. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. mini-SWE-agent harness; exact harness version and step budget unpublished. Separate from other vendors’ coding-agent configurations. Both figure themes were visually checked; table value matches the labeled figure.

47.0 score · raw 47 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. Lite population/revision and scoring formula unspecified; not Terminal-Bench full, v2.1 or v4.0.

66.0 score · raw 66 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. Dataset judge score, explicitly not Elo. Judge rubric, population and score normalization unpublished.

29.7 score · raw 29.7 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. Public split, explicitly not the official evaluation set; exact revision/population and strict-versus-partial reward unknown.

55.7% · raw 55.7 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. Explicit Plus task, not BrowseComp. Exact corpus revision, harness and search budget unpublished. Both figure themes were visually checked; table value matches the labeled figure.

59.8% · raw 59.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown.

67.3% · raw 67.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. No v1.1 revision or exact context budget is established by this unversioned vendor label.

5.8 score · raw 5.8 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. PDFs are OCR-ed in advance; no native image input. Strict all-pass versus partial rubric reward and exact population unspecified.

92.9 score · raw 92.9 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: KRAFTON publisher protocol. KRAFTON Raon-1 mini preview card. Publisher-reported evaluation, not directly verified independent AA results. Source says judge and user simulator gpt-5.6-luna for all evaluations. Exact run dates, serving precision, benchmark inference budgets and peer checkpoint hashes unpublished. Card recommends thinking, temperature 1.0/top-p 0.95/top-k 20, but does not explicitly restate these as the executed settings of every benchmark; effort remains unknown. Unreleased internal Korean colloquial-dialogue test; rubric/normalization, question count and uncertainty unpublished. Both figure themes were visually checked; table value matches the labeled figure.

4.76 tokens/step · raw 4.76 tokens/step

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

296.3 tokens/s · raw 296.3 tokens/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

646.3 tokens/s · raw 646.3 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1110.4 tokens/s · raw 1110.4 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1836.3 tokens/s · raw 1836.3 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1528.9 tokens/s · raw 1528.9 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2619.3 tokens/s · raw 2619.3 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2.18 × · raw 2.18 ×

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.65 × · raw 1.65 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.71 × · raw 1.71 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

4.43 tokens/step · raw 4.43 tokens/step

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

308.0 tokens/s · raw 308 tokens/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

666.6 tokens/s · raw 666.6 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1496.1 tokens/s · raw 1496.1 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2511.3 tokens/s · raw 2511.3 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

3082.5 tokens/s · raw 3082.5 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

5166.9 tokens/s · raw 5166.9 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2.16 × · raw 2.16 ×

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.68 × · raw 1.68 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.68 × · raw 1.68 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

3.96 tokens/step · raw 3.96 tokens/step

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

305.4 tokens/s · raw 305.4 tokens/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

607.9 tokens/s · raw 607.9 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1470.8 tokens/s · raw 1470.8 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2269.6 tokens/s · raw 2269.6 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2793.1 tokens/s · raw 2793.1 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

4596.4 tokens/s · raw 4596.4 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.99 × · raw 1.99 ×

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.54 × · raw 1.54 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.65 × · raw 1.65 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

3.85 tokens/step · raw 3.85 tokens/step

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

307.0 tokens/s · raw 307 tokens/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

604.1 tokens/s · raw 604.1 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1450.6 tokens/s · raw 1450.6 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2197.2 tokens/s · raw 2197.2 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2956.2 tokens/s · raw 2956.2 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

5017.9 tokens/s · raw 5017.9 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.97 × · raw 1.97 ×

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.51 × · raw 1.51 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.70 × · raw 1.7 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

3.24 tokens/step · raw 3.24 tokens/step

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

304.9 tokens/s · raw 304.9 tokens/s

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

570.2 tokens/s · raw 570.2 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1477.1 tokens/s · raw 1477.1 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2038.2 tokens/s · raw 2038.2 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

2521.3 tokens/s · raw 2521.3 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: Autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

3935.4 tokens/s · raw 3935.4 tokens/s

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.87 × · raw 1.87 ×

Headline · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 1. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.38 × · raw 1.38 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 8. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.

1.56 × · raw 1.56 ×

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-25

Harness: DFlash2 versus autoregressive at concurrency 32. One NVIDIA B300, MXFP8 target, vLLM 0.28.0, thinking, T1/top-p.95/top-k20, max4096new tokens;128prompts (80 MT-Bench). DFlash2 block8/seven draft tokens, probabilistic draft sampling. Throughput uses total output tokens/end-to-end wall time at the named concurrency; acceptance is mean tokens/verification step at C1. Never an API endpoint speed or BF16 capability score. Default throughput observation is base autoregressive C1, not the optional drafter’s maximum.