H2O-Lightning-31B Decision Model Results

H2O-Lightning-31B

Explore H2O-Lightning-31B: exact decision configuration, published specifications, source-linked benchmarks and API tariffs when available.

Published specifications

Provider: H2O.ai
Version: v1.2.4 (checkpoint step 22276)
System type: Trained checkpoint
Access: Open weights
License: H2O Model Non-Commercial Licence v1.1 (gated: accept the licence on the model page)
Inputs: text, image
Decision types: Choice, Yes / no, Score, Box
Published size: 31.27B
Active parameters: Not published
Context / input limit: 131,072 tokens
Announced: 2026-10-20

Model card & announcement · Website

Model notes

Decision model from H2O.ai: one forward pass and one output token per decision, with a probability for every option. A fine-tune by its developer, not a third-party quantization. Announced date is the Hugging Face repository creation date; no separate announcement was found. Built on google/gemma-4-31B-it. Repository created 20 October 2026; older omission surfaced by the family's 1 November releases. Benchmark Heaven board rows for other releases and the community Decision Index row name earlier releases, so they are not stored here.

Checkpoint revision: f28212fc0d450226675b15f053cee67cda504108

← All decision models · Compare this decision system →

9 recorded observations. Published results retain their source and setup; no Examenos evaluation was run. Missing results are not zero. How to read these comparisons →

Official / vendor · H2O.ai model cards (vendor measured)

H2O-Lightning-31B · Official / vendor · H2O.ai model cards (vendor measured)
Evaluation	Result	Evidence
H2O Lightning cards · JevBench 231 public items	93.1%	Setup & source Correct answers on the 231 public JevBench items, read through JevBench's own CLI in the setup each card names for JevBench. Release v1.2.4; two-way profile, the JevBench runbook, one NVIDIA L40S. 215 of 231 correct on the card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 231 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · 31B Decision Index profile · JevBench 231 public items	92.6%	Setup & source H2O-Lightning-31B decision-index profile, with its conditional re-read, on the JevBench items. Release v1.2.4; decision-index profile, one NVIDIA L40S. 214 of 231 correct on the card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 231 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · One-way reading · JevBench 231 public items	93.5%	Setup & source The same JevBench items read one-way, the fallback on stock vLLM. Release v1.2.4; stock vLLM one-way, one H100 80GB, on a pre-release build with the same weights. 216 of 231 correct on the card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 231 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · Decision Index 0.3 public text index	67.75	Setup & source The kit's chance-corrected index on the public text suite (140,178 scored rows), in the setup each card names for the Decision Index. Release v1.2.4; decision-index profile, full public run on six H100 80GB at MAX_LEN 81,920; complete, none refused. Printed on the model card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 140178 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · Decision Index 0.3 public vision index	76.59	Setup & source The kit's index on the public vision suite (12,210 rows), in the setup each card names for the Decision Index. Release v1.2.4; decision-index profile with the 1120 image re-read, one NVIDIA RTX PRO 6000. The card's two-way column also prints 75.78 on one L40S. Printed on the model card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 12210 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · 31B two-way, one read · Decision Index 0.3 public text index	66.69	Setup & source H2O-Lightning-31B two-way with no re-read, on one RTX PRO 6000. Release v1.2.4; two-way, one read per question, one NVIDIA RTX PRO 6000. Printed on the model card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 140178 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · 31B two-way, one read · Decision Index 0.3 public vision index	75.64	Setup & source H2O-Lightning-31B two-way reading each image once, on one RTX PRO 6000. Release v1.2.4; two-way, each image read once, one NVIDIA RTX PRO 6000. Printed on the model card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 12210 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · One-way reading · Decision Index 0.3 public text index	65.29	Setup & source The public text suite read one-way, the fallback on stock vLLM. Release v1.2.4; stock vLLM one-way, one H100 80GB. Printed on the model card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 140178 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · One-way reading · Decision Index 0.3 public vision index	75.21	Setup & source The public vision suite read one-way, the fallback on stock vLLM. Release v1.2.4; stock vLLM one-way, one H100 80GB. Printed on the model card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 12210 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source