Examenos

HF text 0.4.1 · Full score

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Public index

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4.1 · Private new-domain component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Raw public index

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Public breadth index

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Equated private same-skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4.1 · Equated private new domains

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Full contribution · public

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Full contribution · same_skills

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4.1 · Full contribution · new_domains

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Knowledge · raw

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6713, "skill": 0.5998, "combined": 0.5166, "coverage": 1, "pending": 0, "n": 11} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 11

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Knowledge · skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6713, "skill": 0.5998, "combined": 0.5166, "coverage": 1, "pending": 0, "n": 11} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 11

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Knowledge · combined

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6713, "skill": 0.5998, "combined": 0.5166, "coverage": 1, "pending": 0, "n": 11} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 11

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Language · raw

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.7931, "skill": 0.7066, "combined": 0.636, "coverage": 1, "pending": 0, "n": 9} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 9

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Language · skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.7931, "skill": 0.7066, "combined": 0.636, "coverage": 1, "pending": 0, "n": 9} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 9

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Language · combined

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.7931, "skill": 0.7066, "combined": 0.636, "coverage": 1, "pending": 0, "n": 9} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 9

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Retrieval · raw

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7632, "skill": 0.691, "combined": 0.6772, "coverage": 1, "pending": 0, "n": 6} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Retrieval · skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7632, "skill": 0.691, "combined": 0.6772, "coverage": 1, "pending": 0, "n": 6} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Retrieval · combined

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7632, "skill": 0.691, "combined": 0.6772, "coverage": 1, "pending": 0, "n": 6} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Tools · raw

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.8965, "skill": 0.8803, "combined": 0.8365, "coverage": 1, "pending": 0, "n": 5} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Tools · skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.8965, "skill": 0.8803, "combined": 0.8365, "coverage": 1, "pending": 0, "n": 5} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Tools · combined

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.8965, "skill": 0.8803, "combined": 0.8365, "coverage": 1, "pending": 0, "n": 5} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Arts · raw

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7739, "skill": 0.6353, "combined": 0.6175, "coverage": 1, "pending": 0, "n": 6} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Arts · skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7739, "skill": 0.6353, "combined": 0.6175, "coverage": 1, "pending": 0, "n": 6} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Arts · combined

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7739, "skill": 0.6353, "combined": 0.6175, "coverage": 1, "pending": 0, "n": 6} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 6

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · ECE

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 34411

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Brier

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 34411

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 34411

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Mean chosen confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 34411

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Wrong among answers at ≥95% confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 34411

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Knowledge · ECE

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Knowledge · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Knowledge · Mean chosen confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Language · ECE

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Language · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Language · Mean chosen confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Retrieval · ECE

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Retrieval · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Retrieval · Mean chosen confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Tools · ECE

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Tools · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Tools · Mean chosen confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Arts · ECE

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Arts · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Arts · Mean chosen confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0–0.1 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0–0.1 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0–0.1 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.1–0.2 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.1. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.1–0.2 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.1. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.1–0.2 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.1. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.2–0.3 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.2. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.2–0.3 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.2. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.2–0.3 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.2. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.3–0.4 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.3. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.3–0.4 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.3. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.3–0.4 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.3. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.4–0.5 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.4. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.4–0.5 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.4. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.4–0.5 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.4. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.5–0.6 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.5. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.5–0.6 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.5. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.5–0.6 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.5. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.6–0.7 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.6. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.6–0.7 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.6. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.6–0.7 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.6. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.7–0.8 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.7. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.7–0.8 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.7. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.7–0.8 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.7. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.8–0.9 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.8. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.8–0.9 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.8. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.8–0.9 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.8. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.9–1.0 · Sample weight

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.9. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.9–1.0 · Mean confidence

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.9. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text calibration 0.4 · Confidence bin 0.9–1.0 · Sample accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.9. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Local median latency

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Latency is measured on its own, after scoring: every entrant runs the same 750 rows (stratified by benchmark and prompt length across the 0.2 scope, plus 10 untimed warm-up rows) as one process sending one request at a time on 1 x NVIDIA RTX PRO 6000, using the fastest inference path the model or its code supports. The figure is the wall time of one row, request in to all answers out, measured the same way for in-process and HTTP engines; board medians and p95s come from this run, not from the scoring runs. Entrant-specific serving/hardware caveats in configuration take precedence; HTTPS Jev is not isolated local inference. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Local p95 latency

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Latency is measured on its own, after scoring: every entrant runs the same 750 rows (stratified by benchmark and prompt length across the 0.2 scope, plus 10 untimed warm-up rows) as one process sending one request at a time on 1 x NVIDIA RTX PRO 6000, using the fastest inference path the model or its code supports. The figure is the wall time of one row, request in to all answers out, measured the same way for in-process and HTTP engines; board medians and p95s come from this run, not from the scoring runs. Entrant-specific serving/hardware caveats in configuration take precedence; HTTPS Jev is not isolated local inference. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Local mean latency

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Latency is measured on its own, after scoring: every entrant runs the same 750 rows (stratified by benchmark and prompt length across the 0.2 scope, plus 10 untimed warm-up rows) as one process sending one request at a time on 1 x NVIDIA RTX PRO 6000, using the fastest inference path the model or its code supports. The figure is the wall time of one row, request in to all answers out, measured the same way for in-process and HTTP engines; board medians and p95s come from this run, not from the scoring runs. Entrant-specific serving/hardware caveats in configuration take precedence; HTTPS Jev is not isolated local inference. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · case exact accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9782, "jev": null, "metric": "case exact accuracy", "requests": 1694, "answered": 1694, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 1, "dataset": "BFCL", "short": "BFCL", "metric": "case exact accuracy", "explainer": "Says yes or no for each supplied function: call it for this request? All must be right.", "baseline": {"value": 0.2592, "label": "Random whole-case guesses"}, "lower": false, "range": null, "cases": 1694, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2305.15334"}, {"kind": "code", "url": "https://github.com/gorilla-llm/gorilla"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1694

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9782, "skill": 0.9705, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9534, "private_eq": 0.9651} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1694

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9782, "skill": 0.9705, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9534, "private_eq": 0.9651} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1694

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9782, "skill": 0.9705, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9534, "private_eq": 0.9651} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BFCL · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9782, "skill": 0.9705, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9534, "private_eq": 0.9651} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · nDCG@10

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6981, "jev": null, "metric": "nDCG@10", "requests": 685, "answered": 685, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 2, "dataset": "ToolRet", "short": "ToolRet", "metric": "nDCG@10", "explainer": "Rates how useful each of 32 search-hit tools is for a request; ranked by nDCG@10, on requests where at least one of the 32 is relevant.", "baseline": {"value": 0.1341, "label": "Random ranking, answerable queries", "previous": 0.0937}, "lower": false, "range": null, "cases": 685, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2503.01763"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/mangopy/ToolRet-Queries"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 685

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.6981, "skill": 0.6514, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7035, "private_eq": 0.7453} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 685

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.6981, "skill": 0.6514, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7035, "private_eq": 0.7453} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 685

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.6981, "skill": 0.6514, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7035, "private_eq": 0.7453} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ToolRet · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.6981, "skill": 0.6514, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7035, "private_eq": 0.7453} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8898, "jev": null, "metric": "accuracy", "requests": 508, "answered": 508, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 3, "dataset": "API-Bank", "short": "API-Bank", "metric": "accuracy", "explainer": "Reads an assistant conversation and picks the next API to call from a catalog of 53.", "baseline": {"value": 0.0189, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 508, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2304.08244"}, {"kind": "code", "url": "https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/api-bank"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 508

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8898, "skill": 0.8877, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.9485, "private_eq": 0.9491} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 508

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8898, "skill": 0.8877, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.9485, "private_eq": 0.9491} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 508

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8898, "skill": 0.8877, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.9485, "private_eq": 0.9491} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · API-Bank · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8898, "skill": 0.8877, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.9485, "private_eq": 0.9491} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · macro-F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8726, "jev": null, "metric": "macro-F1", "requests": 3080, "answered": 3080, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 4, "dataset": "BANKING77", "short": "BANKING77", "metric": "macro-F1", "explainer": "Picks the intent of a bank customer's message, e.g. \"How do I locate my card?\", from 77.", "baseline": {"value": 0.0127, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 3080, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2003.04807"}, {"kind": "dataset", "url": "https://github.com/PolyAI-LDN/task-specific-datasets"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3080

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8726, "skill": 0.871, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.8974, "private_eq": 0.8989} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3080

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8726, "skill": 0.871, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.8974, "private_eq": 0.8989} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3080

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8726, "skill": 0.871, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.8974, "private_eq": 0.8989} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BANKING77 · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8726, "skill": 0.871, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.8974, "private_eq": 0.8989} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · macro-F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9465, "jev": null, "metric": "macro-F1", "requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 5, "dataset": "CLINC150+OOS", "short": "CLINC150", "metric": "macro-F1", "explainer": "Reads a short assistant request and picks its intent from 150, or \"out of scope\" when none fits.", "baseline": {"value": 0.006, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 5500, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/clinc/oos-eval"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9465, "skill": 0.9462, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.8345, "private_eq": 0.8364} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9465, "skill": 0.9462, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.8345, "private_eq": 0.8364} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9465, "skill": 0.9462, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.8345, "private_eq": 0.8364} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLINC150+OOS · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9465, "skill": 0.9462, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.8345, "private_eq": 0.8364} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · RouterBench · selected quality (quality objective)

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8007, "jev": null, "metric": "selected quality (quality objective)", "requests": 10000, "answered": 10000, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 6, "dataset": "RouterBench", "short": "RouterBench", "metric": "selected quality (quality objective)", "explainer": "Picks which of 11 AI models should answer a prompt; scored by its real answer.", "baseline": {"value": 0.5735, "label": "Random model selection"}, "lower": false, "range": null, "cases": 5001, "group": "tools", "in_suite": true, "interactive": false, "index_note": "Not in the index: the prompt reveals that GPT-4 is the best route, so the top models all pick GPT-4 and score within a fraction of a point of each other. Scores stay visible.", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2403.12031"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/withmartian/routerbench"}]} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 10000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · RouterBench · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native selected quality (quality objective). Source normalization: {"raw": 0.8007, "skill": 0.5327, "coverage": 1, "random": 0.5735, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 10000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · RouterBench · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native selected quality (quality objective). Source normalization: {"raw": 0.8007, "skill": 0.5327, "coverage": 1, "random": 0.5735, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 10000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · case exact accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9886, "jev": null, "metric": "case exact accuracy", "requests": 88, "answered": 88, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 9, "dataset": "Home appliance simulator", "short": "Home appliances", "metric": "case exact accuracy", "explainer": "Given a simulated home and a command, answers ~18 questions on handling it; all must be right. Duplicate and dev-set rows removed.", "baseline": {"value": 0, "label": "Random whole-case guesses"}, "lower": false, "range": null, "cases": 88, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "code", "url": "https://github.com/apolinario/decision-index/blob/main/decision_index/suite/build/home_appliance.py"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 88

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9886, "skill": 0.9886, "coverage": 1, "random": 0, "tracks": [], "combined": 0.8999, "private_eq": 0.9} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 88

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9886, "skill": 0.9886, "coverage": 1, "random": 0, "tracks": [], "combined": 0.8999, "private_eq": 0.9} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 88

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9886, "skill": 0.9886, "coverage": 1, "random": 0, "tracks": [], "combined": 0.8999, "private_eq": 0.9} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Home appliance simulator · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9886, "skill": 0.9886, "coverage": 1, "random": 0, "tracks": [], "combined": 0.8999, "private_eq": 0.9} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · SGD/SGD-X · macro-F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.4462, "jev": null, "metric": "macro-F1", "requests": 2500, "answered": 2500, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 10, "dataset": "SGD/SGD-X", "short": "SGD", "metric": "macro-F1", "explainer": "Reads a service dialogue so far and picks the intent the user is pursuing now, or NONE.", "baseline": {"value": 0.399, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 2500, "group": "retrieval", "in_suite": true, "interactive": false, "index_note": "Not in the index: build bug, pending re-run. The builder gave every SGD row the whole dialogue, including the turns after the one being classified. Scores stay visible; SGD returns to the index once re-run with the fixed builder.", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/1909.05855"}, {"kind": "dataset", "url": "https://github.com/google-research-datasets/dstc8-schema-guided-dialogue"}]} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · SGD/SGD-X · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.4462, "skill": 0.0785, "coverage": 1, "random": 0.399, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · SGD/SGD-X · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.4462, "skill": 0.0785, "coverage": 1, "random": 0.399, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ContractNLI · macro-F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8279, "jev": null, "metric": "macro-F1", "requests": 123, "answered": 123, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 11, "dataset": "ContractNLI", "short": "ContractNLI", "metric": "macro-F1", "explainer": "Says whether a full NDA supports, contradicts or omits each of 17 fixed statements.", "baseline": {"value": 0.3085, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 123, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2021.findings-emnlp.164"}, {"kind": "dataset", "url": "https://stanfordnlp.github.io/contract-nli/"}]} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 123

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ContractNLI · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8279, "skill": 0.7512, "coverage": 1, "random": 0.3085, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 123

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ContractNLI · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8279, "skill": 0.7512, "coverage": 1, "random": 0.3085, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 123

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · macro-F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7919, "jev": null, "metric": "macro-F1", "requests": 3200, "answered": 3200, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 12, "dataset": "ANLI", "short": "ANLI", "metric": "macro-F1", "explainer": "Decides whether a claim follows from, contradicts, or is neutral to a premise written to fool AI models.", "baseline": {"value": 0.3324, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 3200, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/1910.14599"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/facebook/anli"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3200

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7919, "skill": 0.6883, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5775, "private_eq": 0.7205} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3200

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7919, "skill": 0.6883, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5775, "private_eq": 0.7205} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3200

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7919, "skill": 0.6883, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5775, "private_eq": 0.7205} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ANLI · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7919, "skill": 0.6883, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5775, "private_eq": 0.7205} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.993, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 20, "dataset": "BPoMP", "short": "BPoMP", "metric": "accuracy", "explainer": "Given two versions of a limerick, one with its rhyme, meter or sense broken, picks the original.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 811, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://zenodo.org/records/7299879"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9929, "skill": 0.9858, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9939, "private_eq": 0.9965} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9929, "skill": 0.9858, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9939, "private_eq": 0.9965} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9929, "skill": 0.9858, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9939, "private_eq": 0.9965} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BPoMP · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9929, "skill": 0.9858, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9939, "private_eq": 0.9965} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6792, "jev": null, "metric": "accuracy", "requests": 2628, "answered": 2628, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 21, "dataset": "Humicroedit", "short": "Humicroedit", "metric": "accuracy", "explainer": "Given two one-word edits of a news headline, picks the one people rated funnier.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2628, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/1906.00274"}, {"kind": "dataset", "url": "https://cs.rochester.edu/u/nhossain/humicroedit.html"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2628

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6792, "skill": 0.3584, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.3269, "private_eq": 0.6668} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2628

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6792, "skill": 0.3584, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.3269, "private_eq": 0.6668} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2628

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6792, "skill": 0.3584, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.3269, "private_eq": 0.6668} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Humicroedit · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6792, "skill": 0.3584, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.3269, "private_eq": 0.6668} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7835, "jev": null, "metric": "accuracy", "requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 22, "dataset": "POP909-CL", "short": "POP909", "metric": "accuracy", "explainer": "Given the notes around a beat in a pop song (not audio), picks the chord playing from 129.", "baseline": {"value": 0.0078, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2000, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2510.06528"}, {"kind": "dataset", "url": "https://github.com/AndyWeasley2004/POP909-CL-Dataset"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7803, "skill": 0.7786, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5922, "private_eq": 0.599} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7803, "skill": 0.7786, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5922, "private_eq": 0.599} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7803, "skill": 0.7786, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5922, "private_eq": 0.599} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · POP909-CL · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7803, "skill": 0.7786, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5922, "private_eq": 0.599} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · cfcolor · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.686, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 23, "dataset": "cfcolor", "short": "cfcolor", "metric": "accuracy", "explainer": "Given a person's earlier ratings of color palettes, predicts which of two new palettes they rated higher.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 5000, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://www.dgp.toronto.edu/~donovan/cfcolor/"}]} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · cfcolor · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6999, "skill": 0.3999, "coverage": 1, "random": 0.5, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · cfcolor · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6999, "skill": 0.3999, "coverage": 1, "random": 0.5, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MMLU · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8922, "jev": null, "metric": "accuracy", "requests": 14033, "answered": 14033, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 24, "dataset": "MMLU", "short": "MMLU", "metric": "accuracy", "explainer": "Four-option questions across 57 school and professional subjects, from law to algebra.", "baseline": {"value": 0.25, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 14042, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/cais/mmlu"}]} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 14033

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MMLU · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8922, "skill": 0.8563, "coverage": 1, "random": 0.25, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 14033

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MMLU · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8922, "skill": 0.8563, "coverage": 1, "random": 0.25, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 14033

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6327, "jev": null, "metric": "accuracy", "requests": 196, "answered": 196, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 25, "dataset": "GPQA Diamond", "short": "GPQA Diamond", "metric": "accuracy", "explainer": "198 hard four-option science questions written by PhD experts in physics, chemistry and biology.", "baseline": {"value": 0.25, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 198, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/idavidrein/gpqa"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 196

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6327, "skill": 0.5102, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.5584, "combined": 0.4054} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 196

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6327, "skill": 0.5102, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.5584, "combined": 0.4054} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 196

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6327, "skill": 0.5102, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.5584, "combined": 0.4054} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GPQA Diamond · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6327, "skill": 0.5102, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.5584, "combined": 0.4054} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Easy · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.992, "jev": null, "metric": "accuracy", "requests": 2376, "answered": 2376, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 26, "dataset": "ARC-Easy", "short": "ARC-Easy", "metric": "accuracy", "explainer": "Grade-school science multiple choice, the easier set.", "baseline": {"value": 0.2502, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2376, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/allenai/ai2_arc"}]} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2376

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Easy · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.992, "skill": 0.9893, "coverage": 1, "random": 0.2502, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2376

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Easy · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.992, "skill": 0.9893, "coverage": 1, "random": 0.2502, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2376

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Challenge · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9812, "jev": null, "metric": "accuracy", "requests": 1172, "answered": 1172, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 27, "dataset": "ARC-Challenge", "short": "ARC-Challenge", "metric": "accuracy", "explainer": "Grade-school science multiple choice, the harder set that simple retrieval gets wrong.", "baseline": {"value": 0.2502, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 1172, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/allenai/ai2_arc"}]} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1172

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Challenge · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9812, "skill": 0.9749, "coverage": 1, "random": 0.2502, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1172

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ARC-Challenge · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9812, "skill": 0.9749, "coverage": 1, "random": 0.2502, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1172

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.929, "jev": null, "metric": "accuracy", "requests": 1267, "answered": 1267, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 28, "dataset": "WinoGrande", "short": "WinoGrande", "metric": "accuracy", "explainer": "Fills a blank with one of two options using common sense, e.g. who \"got the easier cases\".", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 1267, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/allenai/winogrande"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1267

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.929, "skill": 0.858, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.5824, "private_eq": 0.7931} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1267

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.929, "skill": 0.858, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.5824, "private_eq": 0.7931} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1267

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.929, "skill": 0.858, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.5824, "private_eq": 0.7931} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · WinoGrande · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.929, "skill": 0.858, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.5824, "private_eq": 0.7931} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9616, "jev": null, "metric": "accuracy", "requests": 10042, "answered": 10042, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 29, "dataset": "HellaSwag", "short": "HellaSwag", "metric": "accuracy", "explainer": "Picks the most sensible of four continuations of an everyday scene.", "baseline": {"value": 0.25, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 10042, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/Rowan/hellaswag"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 10042

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9616, "skill": 0.9488, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 10042

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9616, "skill": 0.9488, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 10042

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9616, "skill": 0.9488, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HellaSwag · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9616, "skill": 0.9488, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · accuracy (4- and 10-choice)

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8908, "jev": 0.7301, "metric": "accuracy (4- and 10-choice)", "requests": 2638, "answered": 2638, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 30, "dataset": "GSM8K", "short": "GSM8K", "metric": "accuracy", "explainer": "Picks a math word problem's answer from 4, and separately 10, numbers, with no working. Rebuilt for v0.3.", "baseline": {"value": 0.175, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2638, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/openai/gsm8k"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2638

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8908, "skill": 0.8703, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.9242, "random": 0.25}, {"track": "10-choice", "raw": 0.8575, "random": 0.1}], "combined": 0.8313, "private_eq": 0.8616} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2638

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8908, "skill": 0.8703, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.9242, "random": 0.25}, {"track": "10-choice", "raw": 0.8575, "random": 0.1}], "combined": 0.8313, "private_eq": 0.8616} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2638

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8908, "skill": 0.8703, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.9242, "random": 0.25}, {"track": "10-choice", "raw": 0.8575, "random": 0.1}], "combined": 0.8313, "private_eq": 0.8616} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8908, "skill": 0.8703, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.9242, "random": 0.25}, {"track": "10-choice", "raw": 0.8575, "random": 0.1}], "combined": 0.8313, "private_eq": 0.8616} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · 4-choice

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "4-choice", "raw": 0.9242, "random": 0.25} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · GSM8K · 10-choice

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "10-choice", "raw": 0.8575, "random": 0.1} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.2942, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 31, "dataset": "ChessBench", "short": "ChessBench", "metric": "accuracy", "explainer": "Given a chess position and every legal move, picks the strongest one; tied best moves count.", "baseline": {"value": 0.0819, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 5000, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2402.04494"}, {"kind": "code", "url": "https://github.com/google-deepmind/searchless_chess"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.2942, "skill": 0.2313, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.0766, "private_eq": 0.161} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.2942, "skill": 0.2313, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.0766, "private_eq": 0.161} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.2942, "skill": 0.2313, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.0766, "private_eq": 0.161} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ChessBench · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.2942, "skill": 0.2313, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.0766, "private_eq": 0.161} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7699, "jev": null, "metric": "accuracy", "requests": 752, "answered": 752, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 32, "dataset": "MuSR", "short": "MuSR", "metric": "accuracy", "explainer": "Reads a long story and answers one question needing clues from across it, e.g. who the murderer is.", "baseline": {"value": 0.371, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 752, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2310.16049"}, {"kind": "dataset", "url": "https://github.com/Zayne-sprague/MuSR"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 752

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7699, "skill": 0.6342, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.4518, "private_eq": 0.6585} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 752

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7699, "skill": 0.6342, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.4518, "private_eq": 0.6585} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 752

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7699, "skill": 0.6342, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.4518, "private_eq": 0.6585} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MuSR · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7699, "skill": 0.6342, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.4518, "private_eq": 0.6585} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · case exact accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.3333, "jev": null, "metric": "case exact accuracy", "requests": 1650, "answered": 1650, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 33, "dataset": "SATA-Bench", "short": "SATA-Bench", "metric": "case exact accuracy", "explainer": "Given a passage, a question and candidate answers, says yes or no to each; all must be right.", "baseline": {"value": 0.0131, "label": "Random whole-case guesses"}, "lower": false, "range": null, "cases": 1650, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/sata-bench/sata-bench"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1650

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.3333, "skill": 0.3245, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.4204, "private_eq": 0.4335} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1650

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.3333, "skill": 0.3245, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.4204, "private_eq": 0.4335} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1650

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.3333, "skill": 0.3245, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.4204, "private_eq": 0.4335} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · SATA-Bench · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.3333, "skill": 0.3245, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.4204, "private_eq": 0.4335} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · nDCG@10

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.4694, "jev": null, "metric": "nDCG@10", "requests": 220, "answered": 220, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 36, "dataset": "BRIGHT", "short": "BRIGHT", "metric": "nDCG@10", "explainer": "Rates how helpful each of 32 search-hit documents is for a hard question; ranked by nDCG@10, on questions where at least one of the 32 is relevant.", "baseline": {"value": 0.116, "label": "Random ranking, answerable queries", "previous": 0.0448}, "lower": false, "range": null, "cases": 220, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2407.12883"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/xlangai/BRIGHT"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 220

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.4694, "skill": 0.3998, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4023, "private_eq": 0.4768} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 220

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.4694, "skill": 0.3998, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4023, "private_eq": 0.4768} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 220

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.4694, "skill": 0.3998, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4023, "private_eq": 0.4768} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BRIGHT · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.4694, "skill": 0.3998, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4023, "private_eq": 0.4768} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · macro-F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.5722, "jev": null, "metric": "macro-F1", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 37, "dataset": "Amazon ESCI", "short": "Amazon ESCI", "metric": "macro-F1", "explainer": "Labels a product as an exact match, substitute, complement or irrelevant to a shopping query.", "baseline": {"value": 0.2027, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 5000, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/amazon-science/esci-data"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.5722, "skill": 0.4635, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.4661, "private_eq": 0.5784} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.5722, "skill": 0.4635, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.4661, "private_eq": 0.5784} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.5722, "skill": 0.4635, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.4661, "private_eq": 0.5784} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Amazon ESCI · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.5722, "skill": 0.4635, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.4661, "private_eq": 0.5784} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · per-review F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.4688, "jev": null, "metric": "per-review F1", "requests": 1565, "answered": 1565, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 38, "dataset": "ACOS", "short": "ACOS", "metric": "per-review F1", "explainer": "Reads a restaurant or laptop review and says yes/no for each aspect-and-sentiment pair; scored by F1 per review, averaged over reviews.", "baseline": {"value": 0.031, "label": "Always yes · per-review F1", "previous": 0}, "lower": false, "range": null, "cases": 400, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2021.acl-long.29"}, {"kind": "dataset", "url": "https://github.com/NUSTM/ACOS"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1565

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.4688, "skill": 0.4518, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.6501, "private_eq": 0.6638} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1565

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.4688, "skill": 0.4518, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.6501, "private_eq": 0.6638} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1565

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.4688, "skill": 0.4518, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.6501, "private_eq": 0.6638} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · ACOS · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.4688, "skill": 0.4518, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.6501, "private_eq": 0.6638} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · macro-F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.942, "jev": null, "metric": "macro-F1", "requests": 979, "answered": 979, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 39, "dataset": "FinEntity", "short": "FinEntity", "metric": "macro-F1", "explainer": "Says whether a financial news snippet is positive, negative or neutral about a named firm.", "baseline": {"value": 0.3201, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 979, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2023.emnlp-main.956"}, {"kind": "dataset", "url": "https://github.com/yixuantt/FinEntity"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 979

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.942, "skill": 0.9147, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5565, "private_eq": 0.7012} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 979

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.942, "skill": 0.9147, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5565, "private_eq": 0.7012} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 979

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.942, "skill": 0.9147, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5565, "private_eq": 0.7012} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · FinEntity · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.942, "skill": 0.9147, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5565, "private_eq": 0.7012} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Sarcasm F1 · track A, English

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6756, "jev": null, "metric": "Sarcasm F1 \u00b7 track A, English", "requests": 4600, "answered": 4600, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 40, "dataset": "iSarcasmEval", "short": "iSarcasmEval", "metric": "Sarcasm F1 · track A, English", "explainer": "Says yes or no: is this tweet sarcastic? Scored with F1 on sarcastic English tweets (track A); other tracks listed under each result.", "baseline": {"value": 0.2227, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 4600, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2022.semeval-1.111"}, {"kind": "dataset", "url": "https://github.com/iabufarha/iSarcasmEval"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 4600

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.6756, "skill": 0.5826, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.5907, "headline": false}, {"track": "A · English", "score": 0.6756, "headline": true}, {"track": "C · Arabic pairs", "score": 0.96, "headline": false}, {"track": "C · English pairs", "score": 0.99, "headline": false}], "combined": 0.5971, "private_eq": 0.6896} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 4600

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.6756, "skill": 0.5826, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.5907, "headline": false}, {"track": "A · English", "score": 0.6756, "headline": true}, {"track": "C · Arabic pairs", "score": 0.96, "headline": false}, {"track": "C · English pairs", "score": 0.99, "headline": false}], "combined": 0.5971, "private_eq": 0.6896} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 4600

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.6756, "skill": 0.5826, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.5907, "headline": false}, {"track": "A · English", "score": 0.6756, "headline": true}, {"track": "C · Arabic pairs", "score": 0.96, "headline": false}, {"track": "C · English pairs", "score": 0.99, "headline": false}], "combined": 0.5971, "private_eq": 0.6896} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.6756, "skill": 0.5826, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.5907, "headline": false}, {"track": "A · English", "score": 0.6756, "headline": true}, {"track": "C · Arabic pairs", "score": 0.96, "headline": false}, {"track": "C · English pairs", "score": 0.99, "headline": false}], "combined": 0.5971, "private_eq": 0.6896} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · A · Arabic

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "A · Arabic", "score": 0.5907, "headline": false} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · C · Arabic pairs

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "C · Arabic pairs", "score": 0.96, "headline": false} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · iSarcasmEval · C · English pairs

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "C · English pairs", "score": 0.99, "headline": false} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · VAST · macro-F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7757, "jev": null, "metric": "macro-F1", "requests": 3006, "answered": 3006, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 41, "dataset": "VAST", "short": "VAST", "metric": "macro-F1", "explainer": "Says whether a comment is for, against, or neutral on a given topic.", "baseline": {"value": 0.3333, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 3006, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/emilyallaway/zero-shot-stance"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3006

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · VAST · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7757, "skill": 0.6635, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.3293, "private_eq": 0.5573} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3006

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · VAST · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7757, "skill": 0.6635, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.3293, "private_eq": 0.5573} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3006

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · VAST · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7757, "skill": 0.6635, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.3293, "private_eq": 0.5573} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · VAST · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7757, "skill": 0.6635, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.3293, "private_eq": 0.5573} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · macro-F1

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8649, "jev": null, "metric": "macro-F1", "requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 42, "dataset": "NLI4CT", "short": "NLI4CT", "metric": "macro-F1", "explainer": "Says whether clinical-trial report sections support or contradict a statement, some deliberately altered.", "baseline": {"value": 0.486, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 5500, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/ai-systems/Task-2-SemEval-2024"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8649, "skill": 0.7372, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.8965, "private_eq": 0.9469} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8649, "skill": 0.7372, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.8965, "private_eq": 0.9469} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5500

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8649, "skill": 0.7372, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.8965, "private_eq": 0.9469} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · NLI4CT · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8649, "skill": 0.7372, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.8965, "private_eq": 0.9469} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8947, "jev": null, "metric": "accuracy", "requests": 570, "answered": 570, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 43, "dataset": "CRUXEval", "short": "CRUXEval", "metric": "accuracy", "explainer": "Reads a short Python function and an input and picks its output from 2 to 10 candidates, without running it.", "baseline": {"value": 0.3697, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 570, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2401.03065"}, {"kind": "dataset", "url": "https://github.com/facebookresearch/cruxeval"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 570

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8947, "skill": 0.833, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.721, "private_eq": 0.8255} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 570

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8947, "skill": 0.833, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.721, "private_eq": 0.8255} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 570

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8947, "skill": 0.833, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.721, "private_eq": 0.8255} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CRUXEval · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8947, "skill": 0.833, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.721, "private_eq": 0.8255} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9104, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 44, "dataset": "CLadder", "short": "CLadder", "metric": "accuracy", "explainer": "Answers a yes/no question about cause and effect in a small made-up world with stated probabilities.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 5000, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2312.04350"}, {"kind": "dataset", "url": "https://github.com/causalNLP/cladder"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9104, "skill": 0.8208, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7279, "private_eq": 0.865} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9104, "skill": 0.8208, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7279, "private_eq": 0.865} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9104, "skill": 0.8208, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7279, "private_eq": 0.865} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · CLadder · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9104, "skill": 0.8208, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7279, "private_eq": 0.865} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HLE · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.1158, "jev": null, "metric": "accuracy", "requests": 501, "answered": 501, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 45, "dataset": "HLE", "short": "HLE", "metric": "accuracy", "explainer": "Expert-level multiple choice from Humanity's Last Exam, text-only questions, 5 to 26 options.", "baseline": {"value": 0.1641, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 501, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/cais/hle"}], "gold": 1.2} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 501

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HLE · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1158, "skill": 0, "coverage": 1, "random": 0.1641, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 501

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HLE · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1158, "skill": 0, "coverage": 1, "random": 0.1641, "tracks": []} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 501

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6259, "jev": null, "metric": "accuracy", "requests": 1676, "answered": 1676, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 50, "dataset": "Habermas Machine", "short": "Habermas", "metric": "accuracy", "explainer": "Given a group's opinions on a policy question, predicts which candidate statement the group ranked highest.", "baseline": {"value": 0.311, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 1676, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/google-deepmind/habermas_machine"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1676

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6259, "skill": 0.457, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.4063, "private_eq": 0.5949} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1676

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6259, "skill": 0.457, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.4063, "private_eq": 0.5949} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 1676

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6259, "skill": 0.457, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.4063, "private_eq": 0.5949} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · Habermas Machine · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6259, "skill": 0.457, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.4063, "private_eq": 0.5949} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8475, "jev": null, "metric": "accuracy", "requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 56, "dataset": "PhishNChips phishing decisions", "short": "PhishNChips", "metric": "accuracy", "explainer": "Says whether an email and its link are phishing or legitimate.", "baseline": {"value": 0.5, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 2000, "group": "retrieval", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/AreLit/PhishNChips"}, {"kind": "code", "url": "https://github.com/anisselbd/jev-phishing-bench"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8475, "skill": 0.695, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7761, "private_eq": 0.8888} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8475, "skill": 0.695, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7761, "private_eq": 0.8888} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8475, "skill": 0.695, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7761, "private_eq": 0.8888} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · PhishNChips phishing decisions · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8475, "skill": 0.695, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7761, "private_eq": 0.8888} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7565, "jev": null, "metric": "accuracy", "requests": 12032, "answered": 12032, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 57, "dataset": "MMLU-Pro", "short": "MMLU-Pro", "metric": "accuracy", "explainer": "Harder MMLU successor: questions across 14 subjects with up to ten options.", "baseline": {"value": 0.1109, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 12032, "group": "knowledge", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2406.01574"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 12032

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7565, "skill": 0.7261, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6641, "combined": 0.6189} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 12032

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7565, "skill": 0.7261, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6641, "combined": 0.6189} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 12032

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7565, "skill": 0.7261, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6641, "combined": 0.6189} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · MMLU-Pro · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7565, "skill": 0.7261, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6641, "combined": 0.6189} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BBH · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8678, "jev": null, "metric": "accuracy", "requests": 5507, "answered": 5507, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 58, "dataset": "BBH", "short": "BBH", "metric": "accuracy", "explainer": "23 tricky reasoning tasks from BIG-Bench (logic, dates, tracking objects, fallacies), answered with no working.", "baseline": {"value": 0.3101, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 5507, "group": "knowledge", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2210.09261"}, {"kind": "dataset", "url": "https://github.com/suzgunmirac/BIG-Bench-Hard"}], "gold": 1.2, "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5507

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BBH · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8678, "skill": 0.8084, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.5575, "combined": 0.3522} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5507

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BBH · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8678, "skill": 0.8084, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.5575, "combined": 0.3522} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 5507

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BBH · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8678, "skill": 0.8084, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.5575, "combined": 0.3522} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · BBH · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8678, "skill": 0.8084, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.5575, "combined": 0.3522} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · F1 on hallucinated class

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7957, "jev": null, "metric": "F1 on hallucinated class", "requests": 2700, "answered": 2700, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 59, "dataset": "RAGTruth", "short": "RAGTruth", "metric": "F1 on hallucinated class", "explainer": "Says yes or no: does an AI-written answer contain anything its source text doesn't support? Scored with F1.", "baseline": {"value": 0.5177, "label": "Always \"hallucinated\" · F1"}, "lower": false, "range": null, "cases": 2700, "group": "language", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "dataset", "url": "https://github.com/ParticleMedia/RAGTruth"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2700

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.7957, "skill": 0.5764, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.4188, "private_eq": 0.7224} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2700

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.7957, "skill": 0.5764, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.4188, "private_eq": 0.7224} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 2700

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.7957, "skill": 0.5764, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.4188, "private_eq": 0.7224} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · RAGTruth · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.7957, "skill": 0.5764, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.4188, "private_eq": 0.7224} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.871, "jev": null, "metric": "accuracy", "requests": 4000, "answered": 4000, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 61, "dataset": "HoVer", "short": "HoVer", "metric": "accuracy", "explainer": "Decides whether a claim linking two to four Wikipedia articles is supported by their introductions.", "baseline": {"value": 0.5, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 4000, "group": "retrieval", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2011.03088"}, {"kind": "dataset", "url": "https://github.com/hover-nlp/hover"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 4000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.871, "skill": 0.742, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6662, "private_eq": 0.8345} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 4000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.871, "skill": 0.742, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6662, "private_eq": 0.8345} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 4000

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.871, "skill": 0.742, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6662, "private_eq": 0.8345} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · HoVer · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.871, "skill": 0.742, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6662, "private_eq": 0.8345} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9127, "jev": null, "metric": "accuracy", "requests": 3652, "answered": 3652, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 62, "dataset": "When2Call", "short": "When2Call", "metric": "accuracy", "explainer": "Given a question and an assistant's tools, picks the right move: answer, call a tool, ask for details, or decline.", "baseline": {"value": 0.25, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 3652, "group": "tools", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://aclanthology.org/2025.naacl-long.174"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/nvidia/When2Call"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3652

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9127, "skill": 0.8836, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.6315, "private_eq": 0.726} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3652

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9127, "skill": 0.8836, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.6315, "private_eq": 0.726} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 3652

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9127, "skill": 0.8836, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.6315, "private_eq": 0.726} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · When2Call · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9127, "skill": 0.8836, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.6315, "private_eq": 0.726} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · accuracy

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8655, "jev": null, "metric": "accuracy", "requests": 528, "answered": 528, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 64, "dataset": "New Yorker", "short": "New Yorker", "metric": "accuracy", "explainer": "Given a text description of a New Yorker cartoon, picks the caption written for it from five.", "baseline": {"value": 0.2, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 528, "group": "arts", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2209.06293"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/jmhessel/newyorker_caption_contest"}], "private": true} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 528

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · Raw index component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8655, "skill": 0.8319, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.7682, "private_eq": 0.8159} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 528

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · Chance-corrected public skill

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8655, "skill": 0.8319, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.7682, "private_eq": 0.8159} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 528

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · Combined public/private component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8655, "skill": 0.8319, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.7682, "private_eq": 0.8159} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF text 0.4 · New Yorker · Equated private same-skill component

Exact source engine h2o-lightning-31b-v1.3.0; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 31273086512, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/h2oai/h2o-lightning-31b", "code_url": null, "base_model": "google/gemma-4-31B-it", "base_url": "https://huggingface.co/google/gemma-4-31B-it", "weights_repo": null, "served_checkpoint": null, "org": {"id": "h2oai", "name": "H2O.ai", "logo": "data/orgs/h2oai.png"}, "pr": 359, "replaces": "h2o-lightning-31b-v1.1", "new": true}. New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement.

Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #359). Replaces the earlier board entry h2o-lightning-31b-v1.1 (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 70.05, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving (FP8 at load); decision agreement with the submitted rows 99.11% on 831 rows (bar 96%). Served in FP8 at load as in the submitted run, so the quantized agreement bar applies. New weights; a short text question whose first read is under 90% confident is read a second time and the two reads are averaged. The submitted runs were made on an unpublished v1.3.0 release candidate; the published weights reproduce the submitted rows. Its calibration constants were fitted on Decision Index 0.3 public rows (author-declared). Per-benchmark times are not shown: the board run sent requests in batches, so its timings are batch wall-clock. The latency shown is the single-request measurement. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8655, "skill": 0.8319, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.7682, "private_eq": 0.8159} Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Public

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Full

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · CV-Bench · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · CV-Bench · private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · BLINK · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · BLINK · private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · RealWorldQA · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · RealWorldQA · private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · CharXiv · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · CharXiv · private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · InfographicVQA · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · InfographicVQA · private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Mind2Web · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Mind2Web · private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Winoground · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Winoground · private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · KIE (CORD+FUNSD) · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · KIE (CORD+FUNSD) · private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Moderation (Hateful Memes) · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Moderation (Hateful Memes) · private

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · R-Bench-M · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · MMMU-Pro vision · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision calibration 0.4 · acc

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Source calibration sample; no new binning assumptions. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 13695

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision calibration 0.4 · conf

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Source calibration sample; no new binning assumptions. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 13695

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision calibration 0.4 · ece

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Source calibration sample; no new binning assumptions. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 13695

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision calibration 0.4 · over95

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Source calibration sample; no new binning assumptions. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: 13695

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Row latency

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Native image budget · pub

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Heterogeneous native budget 1.6 MP; separate from fixed 1.6MP comparison. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Native image budget · priv

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Heterogeneous native budget 1.6 MP; separate from fixed 1.6MP comparison. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

HF vision 0.4 · Native image budget · full

Exact source engine h2o-lightning-31b-v1.3.0; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. Takes at most 4 images per question; larger questions count as failures.

Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-04T17:52:29Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. Takes at most 4 images per question; larger questions count as failures. Heterogeneous native budget 1.6 MP; separate from fixed 1.6MP comparison. Board pull request 359 says the submitted public runs used an unpublished v1.3.0 release candidate and that the published weights reproduce the submitted rows; its private tests served FP8 at load. The board entry replaces its v1.1 entry; the registry links v1.3.0 to v1.2.4 from the card. The registry pins e2d0dfc, the card revision read.

Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred.

Checked / reported: 2026-11-05 · Samples: Not published

Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified

Correct answers on the 231 public JevBench items, read through JevBench's own CLI in the setup each card names for JevBench.

Release v1.3.0; two-way profile (run_default.sh, jevbench mode), FP8, one NVIDIA RTX PRO 6000; 0 errors.

216 of 231 correct on the card.

H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here.

Checked / reported: 2026-11-05 · Samples: 231

Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02 and 2026-11-05; each result names its release and card revision · Published observations; result reuse terms not specified

H2O-Lightning-31B served by serve.sh with no settings (the Decision Index reading plus its "I don't know" rule) on the 231 public JevBench items.

Release v1.3.0; serve.sh with no settings (aa mode, two-way start-up check verified), FP8, one NVIDIA RTX PRO 6000; 0 errors.

215 of 231 correct on the card.

H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here.

Checked / reported: 2026-11-05 · Samples: 231

Snapshot: 2026-11-05 · Revision: Cards read 2026-11-02 and 2026-11-05; each result names its release and card revision · Published observations; result reuse terms not specified

The kit's chance-corrected index on the public text suite (140,178 scored rows), in the setup each card names for the Decision Index.

Release v1.3.0; decision-index profile (run_decision_index.sh), FP8, one NVIDIA RTX PRO 6000, context 131,072 and two-way window 8,192. The card says it was measured on the release candidate build this package was made from.

Printed on the model card.

H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here.

Checked / reported: 2026-11-05 · Samples: 140178

Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02 and 2026-11-05; each result names its release and card revision · Published observations; result reuse terms not specified

The kit's index on the public vision suite (12,210 rows), in the setup each card names for the Decision Index.

Release v1.3.0; decision-index profile with the 1120 image re-read, FP8, one NVIDIA RTX PRO 6000, at 131,072 / 8,192.

Printed on the model card. In the same comparison the card prints v1.2.4 at 76.56 (+0.03 [-0.81, +0.85], paired over 12,210 rows, within noise); the v1.2.4 card's own 76.59 stays on that entry.

H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here.

Checked / reported: 2026-11-05 · Samples: 12210

Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02 and 2026-11-05; each result names its release and card revision · Published observations; result reuse terms not specified

The v1.3.0 card's weights-only comparison: both releases read through v1.2.4's package (its temperatures, image setting and start-up probes) at MAX_LEN 81,920 and two-way window 16,384, one NVIDIA RTX PRO 6000.

Release v1.3.0; weights read in v1.2.4's comparison package at 81,920 / 16,384, one NVIDIA RTX PRO 6000.

Printed on the v1.3.0 card: +2.27 over v1.2.4, bootstrap interval [+1.84, +2.64] over the kit's case groups (10,000 draws). By area: Knowledge & Reasoning +3.46, Language Understanding +0.84, Retrieval & Classification +2.36, Tools & Automation +1.76, Arts & Human Taste +3.64.

H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here.

Checked / reported: 2026-11-05 · Samples: 140178

Snapshot: 2026-11-05 · Revision: Cards read 2026-11-02 and 2026-11-05; each result names its release and card revision · Published observations; result reuse terms not specified

Box accuracy on RefCOCOg val, 600 referring expressions, excluding board images.

Release v1.3.0; RefCOCOg val, 600 referring expressions, board images excluded; no fitted confidence map applied.

Printed on the model card.

H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here.

Checked / reported: 2026-11-05 · Samples: 600

Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02 and 2026-11-05; each result names its release and card revision · Published observations; result reuse terms not specified

Confidence calibration error against IoU ≥ 0.5, published on a 0–1 scale; lower is better.

Release v1.3.0; confidence calibration error with the readout's own probability, RefCOCOg val, 600 referring expressions.

Published as 0.041 on a 0–1 scale.

H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here.

Checked / reported: 2026-11-05 · Samples: 600

Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02 and 2026-11-05; each result names its release and card revision · Published observations; result reuse terms not specified