Examenos

HF text 0.4.1 · Full score

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034 · Published observations; result reuse terms not specified

HF text 0.4 · Public index

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4.1 · Private new-domain component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034 · Published observations; result reuse terms not specified

HF text 0.4 · Raw public index

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Public breadth index

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Equated private same-skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4.1 · Equated private new domains

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034 · Published observations; result reuse terms not specified

HF text 0.4 · Full contribution · public

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Full contribution · same_skills

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4.1 · Full contribution · new_domains

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034 · Published observations; result reuse terms not specified

HF text 0.4 · Knowledge · raw

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6903, "skill": 0.6232, "combined": 0.5378, "coverage": 1, "pending": 0, "n": 11} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 11

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Knowledge · skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6903, "skill": 0.6232, "combined": 0.5378, "coverage": 1, "pending": 0, "n": 11} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 11

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Knowledge · combined

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6903, "skill": 0.6232, "combined": 0.5378, "coverage": 1, "pending": 0, "n": 11} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 11

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Language · raw

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.8068, "skill": 0.7235, "combined": 0.6257, "coverage": 1, "pending": 0, "n": 9} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 9

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Language · skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.8068, "skill": 0.7235, "combined": 0.6257, "coverage": 1, "pending": 0, "n": 9} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 9

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Language · combined

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.8068, "skill": 0.7235, "combined": 0.6257, "coverage": 1, "pending": 0, "n": 9} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 9

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Retrieval · raw

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.79, "skill": 0.7176, "combined": 0.7321, "coverage": 1, "pending": 0, "n": 6} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Retrieval · skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.79, "skill": 0.7176, "combined": 0.7321, "coverage": 1, "pending": 0, "n": 6} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Retrieval · combined

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.79, "skill": 0.7176, "combined": 0.7321, "coverage": 1, "pending": 0, "n": 6} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Tools · raw

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.8942, "skill": 0.8776, "combined": 0.8447, "coverage": 1, "pending": 0, "n": 5} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Tools · skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.8942, "skill": 0.8776, "combined": 0.8447, "coverage": 1, "pending": 0, "n": 5} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Tools · combined

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.8942, "skill": 0.8776, "combined": 0.8447, "coverage": 1, "pending": 0, "n": 5} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Arts · raw

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7725, "skill": 0.635, "combined": 0.5721, "coverage": 1, "pending": 0, "n": 6} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Arts · skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7725, "skill": 0.635, "combined": 0.5721, "coverage": 1, "pending": 0, "n": 6} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Arts · combined

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7725, "skill": 0.635, "combined": 0.5721, "coverage": 1, "pending": 0, "n": 6} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · ECE

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 34406

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Brier

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 34406

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 34406

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Mean chosen confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 34406

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Wrong among answers at ≥95% confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 34406

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Knowledge · ECE

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Knowledge · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Knowledge · Mean chosen confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Language · ECE

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Language · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Language · Mean chosen confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Retrieval · ECE

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Retrieval · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Retrieval · Mean chosen confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Tools · ECE

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Tools · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Tools · Mean chosen confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Arts · ECE

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Arts · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Arts · Mean chosen confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0–0.1 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0–0.1 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0–0.1 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.1–0.2 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.1. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.1–0.2 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.1. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.1–0.2 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.1. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.2–0.3 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.2. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.2–0.3 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.2. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.2–0.3 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.2. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.3–0.4 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.3. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.3–0.4 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.3. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.3–0.4 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.3. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.4–0.5 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.4. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.4–0.5 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.4. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.4–0.5 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.4. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.5–0.6 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.5. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.5–0.6 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.5. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.5–0.6 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.5. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.6–0.7 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.6. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.6–0.7 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.6. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.6–0.7 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.6. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.7–0.8 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.7. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.7–0.8 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.7. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.7–0.8 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.7. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.8–0.9 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.8. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.8–0.9 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.8. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.8–0.9 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.8. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.9–1.0 · Sample weight

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.9. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.9–1.0 · Mean confidence

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.9. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.9–1.0 · Sample accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.9. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Local median latency

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Latency is measured on its own, after scoring: every entrant runs the same 750 rows (stratified by benchmark and prompt length across the 0.2 scope, plus 10 untimed warm-up rows) as one process sending one request at a time on 1 x NVIDIA RTX PRO 6000, using the fastest inference path the model or its code supports. The figure is the wall time of one row, request in to all answers out, measured the same way for in-process and HTTP engines; board medians and p95s come from this run, not from the scoring runs. Entrant-specific serving/hardware caveats in configuration take precedence; HTTPS Jev is not isolated local inference. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Local p95 latency

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Latency is measured on its own, after scoring: every entrant runs the same 750 rows (stratified by benchmark and prompt length across the 0.2 scope, plus 10 untimed warm-up rows) as one process sending one request at a time on 1 x NVIDIA RTX PRO 6000, using the fastest inference path the model or its code supports. The figure is the wall time of one row, request in to all answers out, measured the same way for in-process and HTTP engines; board medians and p95s come from this run, not from the scoring runs. Entrant-specific serving/hardware caveats in configuration take precedence; HTTPS Jev is not isolated local inference. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Local mean latency

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Latency is measured on its own, after scoring: every entrant runs the same 750 rows (stratified by benchmark and prompt length across the 0.2 scope, plus 10 untimed warm-up rows) as one process sending one request at a time on 1 x NVIDIA RTX PRO 6000, using the fastest inference path the model or its code supports. The figure is the wall time of one row, request in to all answers out, measured the same way for in-process and HTTP engines; board medians and p95s come from this run, not from the scoring runs. Entrant-specific serving/hardware caveats in configuration take precedence; HTTPS Jev is not isolated local inference. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · case exact accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9728, "jev": null, "metric": "case exact accuracy", "requests": 1694, "answered": 1694, "unsupported": 0, "errors": 0, "median_ms": 321.1}. Benchmark contract: {"id": 1, "dataset": "BFCL", "short": "BFCL", "metric": "case exact accuracy", "explainer": "Says yes or no for each supplied function: call it for this request? All must be right.", "baseline": {"value": 0.2592, "label": "Random whole-case guesses"}, "lower": false, "range": null, "cases": 1694, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2305.15334"}, {"kind": "code", "url": "https://github.com/gorilla-llm/gorilla"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1694

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9728, "skill": 0.9633, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9739, "private_eq": 0.9801} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1694

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9728, "skill": 0.9633, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9739, "private_eq": 0.9801} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1694

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9728, "skill": 0.9633, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9739, "private_eq": 0.9801} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9728, "skill": 0.9633, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9739, "private_eq": 0.9801} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · nDCG@10

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.714, "jev": null, "metric": "nDCG@10", "requests": 685, "answered": 685, "unsupported": 0, "errors": 0, "median_ms": 5213.6}. Benchmark contract: {"id": 2, "dataset": "ToolRet", "short": "ToolRet", "metric": "nDCG@10", "explainer": "Rates how useful each of 32 search-hit tools is for a request; ranked by nDCG@10, on requests where at least one of the 32 is relevant.", "baseline": {"value": 0.1341, "label": "Random ranking, answerable queries", "previous": 0.0937}, "lower": false, "range": null, "cases": 685, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2503.01763"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/mangopy/ToolRet-Queries"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 685

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.714, "skill": 0.6697, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7219, "private_eq": 0.7611} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 685

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.714, "skill": 0.6697, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7219, "private_eq": 0.7611} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 685

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.714, "skill": 0.6697, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7219, "private_eq": 0.7611} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.714, "skill": 0.6697, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7219, "private_eq": 0.7611} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8661, "jev": null, "metric": "accuracy", "requests": 508, "answered": 508, "unsupported": 0, "errors": 0, "median_ms": 1642.5}. Benchmark contract: {"id": 3, "dataset": "API-Bank", "short": "API-Bank", "metric": "accuracy", "explainer": "Reads an assistant conversation and picks the next API to call from a catalog of 53.", "baseline": {"value": 0.0189, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 508, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2304.08244"}, {"kind": "code", "url": "https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/api-bank"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 508

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8661, "skill": 0.8635, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.938, "private_eq": 0.9389} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 508

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8661, "skill": 0.8635, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.938, "private_eq": 0.9389} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 508

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8661, "skill": 0.8635, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.938, "private_eq": 0.9389} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8661, "skill": 0.8635, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.938, "private_eq": 0.9389} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · macro-F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9416, "jev": null, "metric": "macro-F1", "requests": 3080, "answered": 3080, "unsupported": 0, "errors": 0, "median_ms": 445.7}. Benchmark contract: {"id": 4, "dataset": "BANKING77", "short": "BANKING77", "metric": "macro-F1", "explainer": "Picks the intent of a bank customer's message, e.g. \"How do I locate my card?\", from 77.", "baseline": {"value": 0.0127, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 3080, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2003.04807"}, {"kind": "dataset", "url": "https://github.com/PolyAI-LDN/task-specific-datasets"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3080

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9416, "skill": 0.9408, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.9198, "private_eq": 0.9207} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3080

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9416, "skill": 0.9408, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.9198, "private_eq": 0.9207} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3080

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9416, "skill": 0.9408, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.9198, "private_eq": 0.9207} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9416, "skill": 0.9408, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.9198, "private_eq": 0.9207} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · macro-F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9607, "jev": null, "metric": "macro-F1", "requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0, "median_ms": 705}. Benchmark contract: {"id": 5, "dataset": "CLINC150+OOS", "short": "CLINC150", "metric": "macro-F1", "explainer": "Reads a short assistant request and picks its intent from 150, or \"out of scope\" when none fits.", "baseline": {"value": 0.006, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 5500, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/clinc/oos-eval"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9607, "skill": 0.9605, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.8763, "private_eq": 0.8774} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9607, "skill": 0.9605, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.8763, "private_eq": 0.8774} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9607, "skill": 0.9605, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.8763, "private_eq": 0.8774} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9607, "skill": 0.9605, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.8763, "private_eq": 0.8774} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RouterBench · selected quality (quality objective)

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7948, "jev": null, "metric": "selected quality (quality objective)", "requests": 10000, "answered": 10000, "unsupported": 0, "errors": 0, "median_ms": 649}. Benchmark contract: {"id": 6, "dataset": "RouterBench", "short": "RouterBench", "metric": "selected quality (quality objective)", "explainer": "Picks which of 11 AI models should answer a prompt; scored by its real answer.", "baseline": {"value": 0.5735, "label": "Random model selection"}, "lower": false, "range": null, "cases": 5001, "group": "tools", "in_suite": true, "interactive": false, "index_note": "Not in the index: the prompt reveals that GPT-4 is the best route, so the top models all pick GPT-4 and score within a fraction of a point of each other. Scores stay visible.", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2403.12031"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/withmartian/routerbench"}]} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 10000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RouterBench · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RouterBench · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native selected quality (quality objective). Source normalization: {"raw": 0.7948, "skill": 0.5189, "coverage": 1, "random": 0.5735, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 10000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RouterBench · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native selected quality (quality objective). Source normalization: {"raw": 0.7948, "skill": 0.5189, "coverage": 1, "random": 0.5735, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 10000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · case exact accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 1, "jev": null, "metric": "case exact accuracy", "requests": 88, "answered": 88, "unsupported": 0, "errors": 0, "median_ms": 991.9}. Benchmark contract: {"id": 9, "dataset": "Home appliance simulator", "short": "Home appliances", "metric": "case exact accuracy", "explainer": "Given a simulated home and a command, answers ~18 questions on handling it; all must be right. Duplicate and dev-set rows removed.", "baseline": {"value": 0, "label": "Random whole-case guesses"}, "lower": false, "range": null, "cases": 88, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "code", "url": "https://github.com/apolinario/decision-index/blob/main/decision_index/suite/build/home_appliance.py"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 88

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 1, "skill": 1, "coverage": 1, "random": 0, "tracks": [], "combined": 0.9479, "private_eq": 0.9475} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 88

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 1, "skill": 1, "coverage": 1, "random": 0, "tracks": [], "combined": 0.9479, "private_eq": 0.9475} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 88

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 1, "skill": 1, "coverage": 1, "random": 0, "tracks": [], "combined": 0.9479, "private_eq": 0.9475} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 1, "skill": 1, "coverage": 1, "random": 0, "tracks": [], "combined": 0.9479, "private_eq": 0.9475} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SGD/SGD-X · macro-F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.4275, "jev": null, "metric": "macro-F1", "requests": 2500, "answered": 2500, "unsupported": 0, "errors": 0, "median_ms": 312}. Benchmark contract: {"id": 10, "dataset": "SGD/SGD-X", "short": "SGD", "metric": "macro-F1", "explainer": "Reads a service dialogue so far and picks the intent the user is pursuing now, or NONE.", "baseline": {"value": 0.399, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 2500, "group": "retrieval", "in_suite": true, "interactive": false, "index_note": "Not in the index: build bug, pending re-run. The builder gave every SGD row the whole dialogue, including the turns after the one being classified. Scores stay visible; SGD returns to the index once re-run with the fixed builder.", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/1909.05855"}, {"kind": "dataset", "url": "https://github.com/google-research-datasets/dstc8-schema-guided-dialogue"}]} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SGD/SGD-X · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SGD/SGD-X · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.4275, "skill": 0.0474, "coverage": 1, "random": 0.399, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SGD/SGD-X · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.4275, "skill": 0.0474, "coverage": 1, "random": 0.399, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ContractNLI · macro-F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8469, "jev": null, "metric": "macro-F1", "requests": 123, "answered": 123, "unsupported": 0, "errors": 0, "median_ms": 911}. Benchmark contract: {"id": 11, "dataset": "ContractNLI", "short": "ContractNLI", "metric": "macro-F1", "explainer": "Says whether a full NDA supports, contradicts or omits each of 17 fixed statements.", "baseline": {"value": 0.3085, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 123, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2021.findings-emnlp.164"}, {"kind": "dataset", "url": "https://stanfordnlp.github.io/contract-nli/"}]} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 123

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ContractNLI · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ContractNLI · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8469, "skill": 0.7786, "coverage": 1, "random": 0.3085, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 123

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ContractNLI · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8469, "skill": 0.7786, "coverage": 1, "random": 0.3085, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 123

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · macro-F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7649, "jev": null, "metric": "macro-F1", "requests": 3200, "answered": 3200, "unsupported": 0, "errors": 0, "median_ms": 133.3}. Benchmark contract: {"id": 12, "dataset": "ANLI", "short": "ANLI", "metric": "macro-F1", "explainer": "Decides whether a claim follows from, contradicts, or is neutral to a premise written to fool AI models.", "baseline": {"value": 0.3324, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 3200, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/1910.14599"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/facebook/anli"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3200

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7649, "skill": 0.6478, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5307, "private_eq": 0.6896} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3200

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7649, "skill": 0.6478, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5307, "private_eq": 0.6896} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3200

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7649, "skill": 0.6478, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5307, "private_eq": 0.6896} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7649, "skill": 0.6478, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5307, "private_eq": 0.6896} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.983, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 130}. Benchmark contract: {"id": 20, "dataset": "BPoMP", "short": "BPoMP", "metric": "accuracy", "explainer": "Given two versions of a limerick, one with its rhyme, meter or sense broken, picks the original.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 811, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://zenodo.org/records/7299879"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9839, "skill": 0.9677, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9577, "private_eq": 0.9786} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9839, "skill": 0.9677, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9577, "private_eq": 0.9786} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9839, "skill": 0.9677, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9577, "private_eq": 0.9786} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9839, "skill": 0.9677, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9577, "private_eq": 0.9786} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6769, "jev": null, "metric": "accuracy", "requests": 2628, "answered": 2628, "unsupported": 0, "errors": 0, "median_ms": 104.8}. Benchmark contract: {"id": 21, "dataset": "Humicroedit", "short": "Humicroedit", "metric": "accuracy", "explainer": "Given two one-word edits of a news headline, picks the one people rated funnier.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2628, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/1906.00274"}, {"kind": "dataset", "url": "https://cs.rochester.edu/u/nhossain/humicroedit.html"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2628

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6769, "skill": 0.3539, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.2768, "private_eq": 0.642} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2628

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6769, "skill": 0.3539, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.2768, "private_eq": 0.642} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2628

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6769, "skill": 0.3539, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.2768, "private_eq": 0.642} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6769, "skill": 0.3539, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.2768, "private_eq": 0.642} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7825, "jev": null, "metric": "accuracy", "requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0, "median_ms": 2182.2}. Benchmark contract: {"id": 22, "dataset": "POP909-CL", "short": "POP909", "metric": "accuracy", "explainer": "Given the notes around a beat in a pop song (not audio), picks the chord playing from 129.", "baseline": {"value": 0.0078, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2000, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2510.06528"}, {"kind": "dataset", "url": "https://github.com/AndyWeasley2004/POP909-CL-Dataset"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7839, "skill": 0.7822, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5654, "private_eq": 0.5727} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7839, "skill": 0.7822, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5654, "private_eq": 0.5727} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7839, "skill": 0.7822, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5654, "private_eq": 0.5727} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7839, "skill": 0.7822, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5654, "private_eq": 0.5727} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · cfcolor · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7034, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 188.5}. Benchmark contract: {"id": 23, "dataset": "cfcolor", "short": "cfcolor", "metric": "accuracy", "explainer": "Given a person's earlier ratings of color palettes, predicts which of two new palettes they rated higher.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 5000, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://www.dgp.toronto.edu/~donovan/cfcolor/"}]} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · cfcolor · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · cfcolor · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7243, "skill": 0.4487, "coverage": 1, "random": 0.5, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · cfcolor · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7243, "skill": 0.4487, "coverage": 1, "random": 0.5, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8873, "jev": null, "metric": "accuracy", "requests": 14033, "answered": 14033, "unsupported": 0, "errors": 0, "median_ms": 112.2}. Benchmark contract: {"id": 24, "dataset": "MMLU", "short": "MMLU", "metric": "accuracy", "explainer": "Four-option questions across 57 school and professional subjects, from law to algebra.", "baseline": {"value": 0.25, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 14042, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/cais/mmlu"}]} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 14033

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8873, "skill": 0.8497, "coverage": 1, "random": 0.25, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 14033

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8873, "skill": 0.8497, "coverage": 1, "random": 0.25, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 14033

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6071, "jev": null, "metric": "accuracy", "requests": 196, "answered": 196, "unsupported": 0, "errors": 0, "median_ms": 140.7}. Benchmark contract: {"id": 25, "dataset": "GPQA Diamond", "short": "GPQA Diamond", "metric": "accuracy", "explainer": "198 hard four-option science questions written by PhD experts in physics, chemistry and biology.", "baseline": {"value": 0.25, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 198, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/idavidrein/gpqa"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 196

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6071, "skill": 0.4762, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.6737, "combined": 0.5609} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 196

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6071, "skill": 0.4762, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.6737, "combined": 0.5609} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 196

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6071, "skill": 0.4762, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.6737, "combined": 0.5609} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6071, "skill": 0.4762, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.6737, "combined": 0.5609} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Easy · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9928, "jev": null, "metric": "accuracy", "requests": 2376, "answered": 2376, "unsupported": 0, "errors": 0, "median_ms": 96.2}. Benchmark contract: {"id": 26, "dataset": "ARC-Easy", "short": "ARC-Easy", "metric": "accuracy", "explainer": "Grade-school science multiple choice, the easier set.", "baseline": {"value": 0.2502, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2376, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/allenai/ai2_arc"}]} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2376

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Easy · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Easy · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9928, "skill": 0.9904, "coverage": 1, "random": 0.2502, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2376

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Easy · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9928, "skill": 0.9904, "coverage": 1, "random": 0.2502, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2376

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Challenge · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9753, "jev": null, "metric": "accuracy", "requests": 1172, "answered": 1172, "unsupported": 0, "errors": 0, "median_ms": 99.7}. Benchmark contract: {"id": 27, "dataset": "ARC-Challenge", "short": "ARC-Challenge", "metric": "accuracy", "explainer": "Grade-school science multiple choice, the harder set that simple retrieval gets wrong.", "baseline": {"value": 0.2502, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 1172, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/allenai/ai2_arc"}]} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1172

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Challenge · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Challenge · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9753, "skill": 0.9671, "coverage": 1, "random": 0.2502, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1172

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Challenge · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9753, "skill": 0.9671, "coverage": 1, "random": 0.2502, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1172

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9605, "jev": null, "metric": "accuracy", "requests": 1267, "answered": 1267, "unsupported": 0, "errors": 0, "median_ms": 93.5}. Benchmark contract: {"id": 28, "dataset": "WinoGrande", "short": "WinoGrande", "metric": "accuracy", "explainer": "Fills a blank with one of two options using common sense, e.g. who \"got the easier cases\".", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 1267, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/allenai/winogrande"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1267

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9605, "skill": 0.921, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6379, "private_eq": 0.8205} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1267

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9605, "skill": 0.921, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6379, "private_eq": 0.8205} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1267

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9605, "skill": 0.921, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6379, "private_eq": 0.8205} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9605, "skill": 0.921, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6379, "private_eq": 0.8205} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9816, "jev": null, "metric": "accuracy", "requests": 10042, "answered": 10042, "unsupported": 0, "errors": 0, "median_ms": 162.5}. Benchmark contract: {"id": 29, "dataset": "HellaSwag", "short": "HellaSwag", "metric": "accuracy", "explainer": "Picks the most sensible of four continuations of an everyday scene.", "baseline": {"value": 0.25, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 10042, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/Rowan/hellaswag"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 10042

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9816, "skill": 0.9755, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 10042

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9816, "skill": 0.9755, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 10042

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9816, "skill": 0.9755, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9816, "skill": 0.9755, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · accuracy (4- and 10-choice)

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8525, "jev": 0.7301, "metric": "accuracy (4- and 10-choice)", "requests": 2638, "answered": 2638, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 30, "dataset": "GSM8K", "short": "GSM8K", "metric": "accuracy", "explainer": "Picks a math word problem's answer from 4, and separately 10, numbers, with no working. Rebuilt for v0.3.", "baseline": {"value": 0.175, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2638, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/openai/gsm8k"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2638

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8525, "skill": 0.8239, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.8901, "random": 0.25}, {"track": "10-choice", "raw": 0.815, "random": 0.1}], "combined": 0.7453, "private_eq": 0.7914} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2638

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8525, "skill": 0.8239, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.8901, "random": 0.25}, {"track": "10-choice", "raw": 0.815, "random": 0.1}], "combined": 0.7453, "private_eq": 0.7914} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2638

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8525, "skill": 0.8239, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.8901, "random": 0.25}, {"track": "10-choice", "raw": 0.815, "random": 0.1}], "combined": 0.7453, "private_eq": 0.7914} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8525, "skill": 0.8239, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.8901, "random": 0.25}, {"track": "10-choice", "raw": 0.815, "random": 0.1}], "combined": 0.7453, "private_eq": 0.7914} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · 4-choice

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "4-choice", "raw": 0.8901, "random": 0.25} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · 10-choice

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "10-choice", "raw": 0.815, "random": 0.1} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.2656, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 402}. Benchmark contract: {"id": 31, "dataset": "ChessBench", "short": "ChessBench", "metric": "accuracy", "explainer": "Given a chess position and every legal move, picks the strongest one; tied best moves count.", "baseline": {"value": 0.0819, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 5000, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2402.04494"}, {"kind": "code", "url": "https://github.com/google-deepmind/searchless_chess"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.2656, "skill": 0.2001, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.1081, "private_eq": 0.1896} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.2656, "skill": 0.2001, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.1081, "private_eq": 0.1896} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.2656, "skill": 0.2001, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.1081, "private_eq": 0.1896} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.2656, "skill": 0.2001, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.1081, "private_eq": 0.1896} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7992, "jev": null, "metric": "accuracy", "requests": 752, "answered": 752, "unsupported": 0, "errors": 0, "median_ms": 455.5}. Benchmark contract: {"id": 32, "dataset": "MuSR", "short": "MuSR", "metric": "accuracy", "explainer": "Reads a long story and answers one question needing clues from across it, e.g. who the murderer is.", "baseline": {"value": 0.371, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 752, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2310.16049"}, {"kind": "dataset", "url": "https://github.com/Zayne-sprague/MuSR"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 752

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7992, "skill": 0.6808, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.4285, "private_eq": 0.644} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 752

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7992, "skill": 0.6808, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.4285, "private_eq": 0.644} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 752

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7992, "skill": 0.6808, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.4285, "private_eq": 0.644} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7992, "skill": 0.6808, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.4285, "private_eq": 0.644} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · case exact accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.3861, "jev": null, "metric": "case exact accuracy", "requests": 1650, "answered": 1650, "unsupported": 0, "errors": 0, "median_ms": 428.3}. Benchmark contract: {"id": 33, "dataset": "SATA-Bench", "short": "SATA-Bench", "metric": "case exact accuracy", "explainer": "Given a passage, a question and candidate answers, says yes or no to each; all must be right.", "baseline": {"value": 0.0131, "label": "Random whole-case guesses"}, "lower": false, "range": null, "cases": 1650, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/sata-bench/sata-bench"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1650

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.3861, "skill": 0.378, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.4512, "private_eq": 0.4636} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1650

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.3861, "skill": 0.378, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.4512, "private_eq": 0.4636} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1650

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.3861, "skill": 0.378, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.4512, "private_eq": 0.4636} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.3861, "skill": 0.378, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.4512, "private_eq": 0.4636} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · nDCG@10

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.515, "jev": null, "metric": "nDCG@10", "requests": 220, "answered": 220, "unsupported": 0, "errors": 0, "median_ms": 6420.1}. Benchmark contract: {"id": 36, "dataset": "BRIGHT", "short": "BRIGHT", "metric": "nDCG@10", "explainer": "Rates how helpful each of 32 search-hit documents is for a hard question; ranked by nDCG@10, on questions where at least one of the 32 is relevant.", "baseline": {"value": 0.116, "label": "Random ranking, answerable queries", "previous": 0.0448}, "lower": false, "range": null, "cases": 220, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2407.12883"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/xlangai/BRIGHT"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 220

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.515, "skill": 0.4513, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4392, "private_eq": 0.509} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 220

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.515, "skill": 0.4513, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4392, "private_eq": 0.509} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 220

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.515, "skill": 0.4513, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4392, "private_eq": 0.509} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.515, "skill": 0.4513, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4392, "private_eq": 0.509} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · macro-F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6155, "jev": null, "metric": "macro-F1", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 177.8}. Benchmark contract: {"id": 37, "dataset": "Amazon ESCI", "short": "Amazon ESCI", "metric": "macro-F1", "explainer": "Labels a product as an exact match, substitute, complement or irrelevant to a shopping query.", "baseline": {"value": 0.2027, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 5000, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/amazon-science/esci-data"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.6155, "skill": 0.5177, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.6174, "private_eq": 0.6976} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.6155, "skill": 0.5177, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.6174, "private_eq": 0.6976} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.6155, "skill": 0.5177, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.6174, "private_eq": 0.6976} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.6155, "skill": 0.5177, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.6174, "private_eq": 0.6976} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · per-review F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6358, "jev": null, "metric": "per-review F1", "requests": 1565, "answered": 1565, "unsupported": 0, "errors": 0, "median_ms": 3439.8}. Benchmark contract: {"id": 38, "dataset": "ACOS", "short": "ACOS", "metric": "per-review F1", "explainer": "Reads a restaurant or laptop review and says yes/no for each aspect-and-sentiment pair; scored by F1 per review, averaged over reviews.", "baseline": {"value": 0.031, "label": "Always yes · per-review F1", "previous": 0}, "lower": false, "range": null, "cases": 400, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2021.acl-long.29"}, {"kind": "dataset", "url": "https://github.com/NUSTM/ACOS"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1565

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.6358, "skill": 0.6241, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.7286, "private_eq": 0.739} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1565

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.6358, "skill": 0.6241, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.7286, "private_eq": 0.739} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1565

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.6358, "skill": 0.6241, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.7286, "private_eq": 0.739} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.6358, "skill": 0.6241, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.7286, "private_eq": 0.739} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · macro-F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9334, "jev": null, "metric": "macro-F1", "requests": 979, "answered": 979, "unsupported": 0, "errors": 0, "median_ms": 144.1}. Benchmark contract: {"id": 39, "dataset": "FinEntity", "short": "FinEntity", "metric": "macro-F1", "explainer": "Says whether a financial news snippet is positive, negative or neutral about a named firm.", "baseline": {"value": 0.3201, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 979, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2023.emnlp-main.956"}, {"kind": "dataset", "url": "https://github.com/yixuantt/FinEntity"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 979

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9334, "skill": 0.902, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5316, "private_eq": 0.6845} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 979

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9334, "skill": 0.902, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5316, "private_eq": 0.6845} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 979

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9334, "skill": 0.902, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5316, "private_eq": 0.6845} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9334, "skill": 0.902, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5316, "private_eq": 0.6845} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Sarcasm F1 · track A, English

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.5473, "jev": null, "metric": "Sarcasm F1 \u00b7 track A, English", "requests": 4600, "answered": 4600, "unsupported": 0, "errors": 0, "median_ms": 102.7}. Benchmark contract: {"id": 40, "dataset": "iSarcasmEval", "short": "iSarcasmEval", "metric": "Sarcasm F1 · track A, English", "explainer": "Says yes or no: is this tweet sarcastic? Scored with F1 on sarcastic English tweets (track A); other tracks listed under each result.", "baseline": {"value": 0.2227, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 4600, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2022.semeval-1.111"}, {"kind": "dataset", "url": "https://github.com/iabufarha/iSarcasmEval"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 4600

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.5473, "skill": 0.4176, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.6025, "headline": false}, {"track": "A · English", "score": 0.5473, "headline": true}, {"track": "C · Arabic pairs", "score": 0.91, "headline": false}, {"track": "C · English pairs", "score": 0.975, "headline": false}], "combined": 0.4124, "private_eq": 0.5477} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 4600

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.5473, "skill": 0.4176, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.6025, "headline": false}, {"track": "A · English", "score": 0.5473, "headline": true}, {"track": "C · Arabic pairs", "score": 0.91, "headline": false}, {"track": "C · English pairs", "score": 0.975, "headline": false}], "combined": 0.4124, "private_eq": 0.5477} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 4600

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.5473, "skill": 0.4176, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.6025, "headline": false}, {"track": "A · English", "score": 0.5473, "headline": true}, {"track": "C · Arabic pairs", "score": 0.91, "headline": false}, {"track": "C · English pairs", "score": 0.975, "headline": false}], "combined": 0.4124, "private_eq": 0.5477} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.5473, "skill": 0.4176, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.6025, "headline": false}, {"track": "A · English", "score": 0.5473, "headline": true}, {"track": "C · Arabic pairs", "score": 0.91, "headline": false}, {"track": "C · English pairs", "score": 0.975, "headline": false}], "combined": 0.4124, "private_eq": 0.5477} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · A · Arabic

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "A · Arabic", "score": 0.6025, "headline": false} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · C · Arabic pairs

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "C · Arabic pairs", "score": 0.91, "headline": false} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · C · English pairs

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "C · English pairs", "score": 0.975, "headline": false} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · VAST · macro-F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8286, "jev": null, "metric": "macro-F1", "requests": 3006, "answered": 3006, "unsupported": 0, "errors": 0, "median_ms": 138.2}. Benchmark contract: {"id": 41, "dataset": "VAST", "short": "VAST", "metric": "macro-F1", "explainer": "Says whether a comment is for, against, or neutral on a given topic.", "baseline": {"value": 0.3333, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 3006, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/emilyallaway/zero-shot-stance"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3006

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · VAST · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · VAST · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8286, "skill": 0.7428, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.4484, "private_eq": 0.6358} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3006

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · VAST · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8286, "skill": 0.7428, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.4484, "private_eq": 0.6358} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3006

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · VAST · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8286, "skill": 0.7428, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.4484, "private_eq": 0.6358} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · VAST · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8286, "skill": 0.7428, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.4484, "private_eq": 0.6358} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · macro-F1

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8627, "jev": null, "metric": "macro-F1", "requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0, "median_ms": 181.1}. Benchmark contract: {"id": 42, "dataset": "NLI4CT", "short": "NLI4CT", "metric": "macro-F1", "explainer": "Says whether clinical-trial report sections support or contradict a statement, some deliberately altered.", "baseline": {"value": 0.486, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 5500, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/ai-systems/Task-2-SemEval-2024"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8627, "skill": 0.7329, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.9892, "private_eq": 0.994} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8627, "skill": 0.7329, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.9892, "private_eq": 0.994} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8627, "skill": 0.7329, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.9892, "private_eq": 0.994} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8627, "skill": 0.7329, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.9892, "private_eq": 0.994} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8947, "jev": null, "metric": "accuracy", "requests": 570, "answered": 570, "unsupported": 0, "errors": 0, "median_ms": 120.3}. Benchmark contract: {"id": 43, "dataset": "CRUXEval", "short": "CRUXEval", "metric": "accuracy", "explainer": "Reads a short Python function and an input and picks its output from 2 to 10 candidates, without running it.", "baseline": {"value": 0.3697, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 570, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2401.03065"}, {"kind": "dataset", "url": "https://github.com/facebookresearch/cruxeval"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 570

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8947, "skill": 0.833, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.6582, "private_eq": 0.7864} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 570

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8947, "skill": 0.833, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.6582, "private_eq": 0.7864} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 570

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8947, "skill": 0.833, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.6582, "private_eq": 0.7864} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8947, "skill": 0.833, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.6582, "private_eq": 0.7864} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9486, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 130.5}. Benchmark contract: {"id": 44, "dataset": "CLadder", "short": "CLadder", "metric": "accuracy", "explainer": "Answers a yes/no question about cause and effect in a small made-up world with stated probabilities.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 5000, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2312.04350"}, {"kind": "dataset", "url": "https://github.com/causalNLP/cladder"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9486, "skill": 0.8972, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.8511, "private_eq": 0.9259} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9486, "skill": 0.8972, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.8511, "private_eq": 0.9259} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9486, "skill": 0.8972, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.8511, "private_eq": 0.9259} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9486, "skill": 0.8972, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.8511, "private_eq": 0.9259} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HLE · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.1956, "jev": null, "metric": "accuracy", "requests": 501, "answered": 501, "unsupported": 0, "errors": 0, "median_ms": 178.7}. Benchmark contract: {"id": 45, "dataset": "HLE", "short": "HLE", "metric": "accuracy", "explainer": "Expert-level multiple choice from Humanity's Last Exam, text-only questions, 5 to 26 options.", "baseline": {"value": 0.1641, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 501, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/cais/hle"}], "gold": 1.2} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 501

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HLE · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HLE · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1956, "skill": 0.0377, "coverage": 1, "random": 0.1641, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 501

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HLE · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1956, "skill": 0.0377, "coverage": 1, "random": 0.1641, "tracks": []} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 501

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6253, "jev": null, "metric": "accuracy", "requests": 1676, "answered": 1676, "unsupported": 0, "errors": 0, "median_ms": 289.7}. Benchmark contract: {"id": 50, "dataset": "Habermas Machine", "short": "Habermas", "metric": "accuracy", "explainer": "Given a group's opinions on a policy question, predicts which candidate statement the group ranked highest.", "baseline": {"value": 0.311, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 1676, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/google-deepmind/habermas_machine"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1676

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6253, "skill": 0.4562, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.2539, "private_eq": 0.4911} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1676

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6253, "skill": 0.4562, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.2539, "private_eq": 0.4911} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 1676

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6253, "skill": 0.4562, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.2539, "private_eq": 0.4911} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6253, "skill": 0.4562, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.2539, "private_eq": 0.4911} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.802, "jev": null, "metric": "accuracy", "requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0, "median_ms": 357.7}. Benchmark contract: {"id": 56, "dataset": "PhishNChips phishing decisions", "short": "PhishNChips", "metric": "accuracy", "explainer": "Says whether an email and its link are phishing or legitimate.", "baseline": {"value": 0.5, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 2000, "group": "retrieval", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/AreLit/PhishNChips"}, {"kind": "code", "url": "https://github.com/anisselbd/jev-phishing-bench"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.802, "skill": 0.604, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.8885, "private_eq": 0.9444} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.802, "skill": 0.604, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.8885, "private_eq": 0.9444} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.802, "skill": 0.604, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.8885, "private_eq": 0.9444} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.802, "skill": 0.604, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.8885, "private_eq": 0.9444} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7601, "jev": null, "metric": "accuracy", "requests": 12032, "answered": 12032, "unsupported": 0, "errors": 0, "median_ms": 141.5}. Benchmark contract: {"id": 57, "dataset": "MMLU-Pro", "short": "MMLU-Pro", "metric": "accuracy", "explainer": "Harder MMLU successor: questions across 14 subjects with up to ten options.", "baseline": {"value": 0.1109, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 12032, "group": "knowledge", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2406.01574"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 12032

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7601, "skill": 0.7302, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6651, "combined": 0.62} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 12032

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7601, "skill": 0.7302, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6651, "combined": 0.62} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 12032

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7601, "skill": 0.7302, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6651, "combined": 0.62} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7601, "skill": 0.7302, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6651, "combined": 0.62} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BBH · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9237, "jev": null, "metric": "accuracy", "requests": 5507, "answered": 5507, "unsupported": 0, "errors": 0, "median_ms": 115.9}. Benchmark contract: {"id": 58, "dataset": "BBH", "short": "BBH", "metric": "accuracy", "explainer": "23 tricky reasoning tasks from BIG-Bench (logic, dates, tracking objects, fallacies), answered with no working.", "baseline": {"value": 0.3101, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 5507, "group": "knowledge", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2210.09261"}, {"kind": "dataset", "url": "https://github.com/suzgunmirac/BIG-Bench-Hard"}], "gold": 1.2, "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5507

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BBH · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BBH · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9237, "skill": 0.8894, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.535, "combined": 0.3192} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5507

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BBH · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9237, "skill": 0.8894, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.535, "combined": 0.3192} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 5507

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BBH · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9237, "skill": 0.8894, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.535, "combined": 0.3192} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · BBH · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9237, "skill": 0.8894, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.535, "combined": 0.3192} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · F1 on hallucinated class

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8335, "jev": null, "metric": "F1 on hallucinated class", "requests": 2700, "answered": 2700, "unsupported": 0, "errors": 0, "median_ms": 226.2}. Benchmark contract: {"id": 59, "dataset": "RAGTruth", "short": "RAGTruth", "metric": "F1 on hallucinated class", "explainer": "Says yes or no: does an AI-written answer contain anything its source text doesn't support? Scored with F1.", "baseline": {"value": 0.5177, "label": "Always \"hallucinated\" · F1"}, "lower": false, "range": null, "cases": 2700, "group": "language", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "dataset", "url": "https://github.com/ParticleMedia/RAGTruth"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2700

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.8335, "skill": 0.6548, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.3074, "private_eq": 0.6693} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2700

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.8335, "skill": 0.6548, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.3074, "private_eq": 0.6693} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 2700

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.8335, "skill": 0.6548, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.3074, "private_eq": 0.6693} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.8335, "skill": 0.6548, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.3074, "private_eq": 0.6693} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8958, "jev": null, "metric": "accuracy", "requests": 4000, "answered": 4000, "unsupported": 0, "errors": 0, "median_ms": 168}. Benchmark contract: {"id": 61, "dataset": "HoVer", "short": "HoVer", "metric": "accuracy", "explainer": "Decides whether a claim linking two to four Wikipedia articles is supported by their introductions.", "baseline": {"value": 0.5, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 4000, "group": "retrieval", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2011.03088"}, {"kind": "dataset", "url": "https://github.com/hover-nlp/hover"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 4000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8958, "skill": 0.7916, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6435, "private_eq": 0.8233} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 4000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8958, "skill": 0.7916, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6435, "private_eq": 0.8233} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 4000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8958, "skill": 0.7916, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6435, "private_eq": 0.8233} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8958, "skill": 0.7916, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6435, "private_eq": 0.8233} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.908, "jev": null, "metric": "accuracy", "requests": 3652, "answered": 3652, "unsupported": 0, "errors": 0, "median_ms": 250.9}. Benchmark contract: {"id": 62, "dataset": "When2Call", "short": "When2Call", "metric": "accuracy", "explainer": "Given a question and an assistant's tools, picks the right move: answer, call a tool, ask for details, or decline.", "baseline": {"value": 0.25, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 3652, "group": "tools", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://aclanthology.org/2025.naacl-long.174"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/nvidia/When2Call"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3652

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.908, "skill": 0.8773, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.5972, "private_eq": 0.7006} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3652

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.908, "skill": 0.8773, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.5972, "private_eq": 0.7006} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 3652

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.908, "skill": 0.8773, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.5972, "private_eq": 0.7006} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.908, "skill": 0.8773, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.5972, "private_eq": 0.7006} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · accuracy

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8409, "jev": null, "metric": "accuracy", "requests": 528, "answered": 528, "unsupported": 0, "errors": 0, "median_ms": 158.7}. Benchmark contract: {"id": 64, "dataset": "New Yorker", "short": "New Yorker", "metric": "accuracy", "explainer": "Given a text description of a New Yorker cartoon, picks the caption written for it from five.", "baseline": {"value": 0.2, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 528, "group": "arts", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2209.06293"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/jmhessel/newyorker_caption_contest"}], "private": true} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 528

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · median_ms

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · Raw index component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8409, "skill": 0.8011, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.8071, "private_eq": 0.8466} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 528

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · Chance-corrected public skill

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8409, "skill": 0.8011, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.8071, "private_eq": 0.8466} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 528

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · Combined public/private component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8409, "skill": 0.8011, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.8071, "private_eq": 0.8466} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · Equated private same-skill component

Exact source engine vllm-sr-d3-ultra; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 26085330160, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/vllm-sr/d3-ultra", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": "vllm-sr/d3-ultra", "served_checkpoint": "37fee01926f3de396620e794201b074fb427ac2d", "weights_note": null, "org": {"id": "vllm-sr", "name": "vLLM Semantic Router", "logo": "data/orgs/vllm-sr.png"}, "pr": 302, "new": true}. New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X.

Decision Index 0.4.1 (release-v4), source generated 2026-10-26T15:30:58Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-26 PR batch (PR #302). Submitted public 0.3 run adopted (kit rescore 71.57, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.81% on 791 rows (bar 98.5%). A separate model from vLLM-SR d3-turbo v1.1 (27B) and vLLM-SR d3 (27B), which stay on the board. Weights are public on the Hugging Face Hub (Apache-2.0); the lab scored revision 37fee01. Served with the model repo's own engine on an RTX PRO 6000, where the submitted run used an AMD MI325X. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8409, "skill": 0.8011, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.8071, "private_eq": 0.8466} The author submitted public text results on AMD MI325X; the evaluator rescored them and ran private text tests on RTX PRO 6000 with author serving. Lab revision 37fee01926f3de396620e794201b074fb427ac2d; all 12 weight file LFS digests match card revision 52ba9e7a00402f9560c7fba8c039608e7ddee7fb.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Public

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Full

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · CV-Bench · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · CV-Bench · private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · BLINK · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · BLINK · private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · RealWorldQA · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · RealWorldQA · private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · CharXiv · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · CharXiv · private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · InfographicVQA · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · InfographicVQA · private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Mind2Web · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Mind2Web · private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Winoground · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Winoground · private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · KIE (CORD+FUNSD) · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · KIE (CORD+FUNSD) · private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Moderation (Hateful Memes) · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Moderation (Hateful Memes) · private

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · R-Bench-M · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · MMMU-Pro vision · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision calibration 0.4 · acc

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Source calibration sample; no new binning assumptions. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 13695

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision calibration 0.4 · conf

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Source calibration sample; no new binning assumptions. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 13695

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision calibration 0.4 · ece

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Source calibration sample; no new binning assumptions. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 13695

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision calibration 0.4 · over95

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Source calibration sample; no new binning assumptions. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: 13695

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Row latency

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Native image budget · pub

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Heterogeneous native budget 1.6 MP; separate from fixed 1.6MP comparison. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Native image budget · priv

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Heterogeneous native budget 1.6 MP; separate from fixed 1.6MP comparison. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

HF vision 0.4 · Native image budget · full

Exact source engine vllm-sr-d3-ultra; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-10-26T15:31:01Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Heterogeneous native budget 1.6 MP; separate from fixed 1.6MP comparison. This is the separate community vision board; the specific vision execution date and weight revision are not stated in this entry.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731 · Published observations; result reuse terms not specified

SR-Text · mean of six chance-corrected parts

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Text · knowledge and reasoning

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Text · language understanding

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Text · retrieval and classification

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Text · tools and automation

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Text · arts and human taste

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Text · new domains

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · ten-domain mean

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · nine-domain mean without charts

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · Photos · one of a kind

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · Photos · one among several

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · Photos · small objects

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · Photos · groups

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · Art

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · Referring expressions

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · Receipts

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · Documents

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · UI screens

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified

SR-Box · Charts

d3-ultra model files; SR tests published in official d3-ultra card. Peer d3 is v3.1.0; d1-ultra uses an unpinned Liquid API 26 October; Jev uses pinned jev-1.14.0 API 26 October, with 86 invalid text requests and 816 images refused for size counted wrong. Do not infer identical deployment conditions.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed. From figure and card; unsupported values are omitted, not zero.

vLLM Semantic Router own private SR test and run, not an independent leaderboard. SR-Text: 10,411 requests / 73,495 questions; six equally weighted chance-corrected parts, not raw accuracy. SR-Box: 4,778 positive questions among 5,678 total on 4,984 images; IoU >= 0.5; mean across domains, with 900 absent-object questions excluded. Exact runtime/budget and benchmark execution date are not fully disclosed.

Checked / reported: 2026-10-27 · Samples: Not published

Snapshot: 2026-10-27 · Revision: 52ba9e7a00402f9560c7fba8c039608e7ddee7fb · Published observations; result reuse terms not specified