H2O Lightning E4B Decision Model Results

H2O Lightning E4B

Explore H2O Lightning E4B: exact decision configuration, published specifications, source-linked benchmarks and API tariffs when available.

Published specifications

Provider: H2O.ai
Version: v1.0
System type: Trained checkpoint
Access: Open weights
License: H2O Model Non-Commercial Licence v1.1 (gated: accept the licence on the model page)
Inputs: text, image
Decision types: Choice, Yes / no, Score, Box
Published size: 7.94B
Active parameters: Not published
Context / input limit: 131,072 tokens
Announced: 2026-11-01

Model card & announcement · Website

Model notes

Decision model from H2O.ai: one forward pass and one output token per decision, with a probability for every option. A fine-tune by its developer, not a third-party quantization. Announced date is the Hugging Face repository creation date; no separate announcement was found. Built on google/gemma-4-E4B-it; 7,941,100,874 stored parameters include the base model's key/value-sharing tensors. Card speed on one L40S on JevBench-sized requests: 47.62 requests per second at 32 in flight, median 24.5 ms one at a time. First published 1 November 2026.

Checkpoint revision: 7bab8740416f49ec396f05dec9b161ab6c85ae75

← All decision models · Compare this decision system →

8 recorded observations. Published results retain their source and setup; no Examenos evaluation was run. Missing results are not zero. How to read these comparisons →

Official / vendor · H2O.ai model cards (vendor measured)

H2O Lightning E4B · Official / vendor · H2O.ai model cards (vendor measured)
Evaluation	Result	Evidence
H2O Lightning cards · JevBench 231 public items	87.9%	Setup & source Correct answers on the 231 public JevBench items, read through JevBench's own CLI in the setup each card names for JevBench. Release v1.0, one NVIDIA L40S; two-way reading, JevBench mode. 203 of 231 correct on the card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 231 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · One-way reading · JevBench 231 public items	88.3%	Setup & source The same JevBench items read one-way, the fallback on stock vLLM. Release v1.0, one NVIDIA L40S; one-way reading; stated in the card prose. 204 of 231 correct on the card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 231 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · Decision Index 0.3 public text index	59.37	Setup & source The kit's chance-corrected index on the public text suite (140,178 scored rows), in the setup each card names for the Decision Index. Release v1.0, one NVIDIA L40S; Decision Index mode (PROFILE=di). Printed on the model card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 140178 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · Decision Index 0.3 public vision index	59.83	Setup & source The kit's index on the public vision suite (12,210 rows), in the setup each card names for the Decision Index. Release v1.0, one NVIDIA L40S; Decision Index mode (PROFILE=di). Printed on the model card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 12210 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · Calibration error, text rows	4.48 pp	Setup & source Calibration error on Decision Index text rows, published on a 0–1 scale; lower is better. Each result quotes its card's wording. Release v1.0, one NVIDIA L40S; card label: calibration error of the top option, text rows, 10 equal-width bins. Published as 0.0448 on a 0–1 scale. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · RefCOCOg box Acc@0.5	73.2%	Setup & source Box accuracy on RefCOCOg val, 600 referring expressions, excluding board images. Release v1.0, one NVIDIA L40S; RefCOCOg val, 600 referring expressions, board COCO images excluded. Printed on the model card. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 600 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · RefCOCOg box mean IoU	69.07%	Setup & source Mean IoU on RefCOCOg val, 600 referring expressions, excluding board images; shown as a percent. Release v1.0, one NVIDIA L40S; RefCOCOg val, 600 referring expressions, board COCO images excluded. Published as 0.6907 on a 0–1 scale. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 600 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source
H2O Lightning cards · RefCOCOg box confidence calibration error	6.11 pp	Setup & source Confidence calibration error against IoU ≥ 0.5, published on a 0–1 scale; lower is better. Release v1.0, one NVIDIA L40S; confidence against IoU ≥ 0.5, RefCOCOg val. Published as 0.0611 on a 0–1 scale. H2O.ai's own runs, not a leaderboard result: its Decision Index rows use the public 0.3 kit and its own scorer, and the JevBench rows use JevBench's own CLI on the 231 public items. Hardware and serving profile differ by release, as each result states. Never compare with Leanmcp JevBench's board rows, Benchmark Heaven or community Decision Index 0.4 rows. Not ranked here. Checked / reported: 2026-11-02 · Samples: 600 Snapshot: 2026-11-02 · Revision: Cards read 2026-11-02; each result names its release and card revision · Published observations; result reuse terms not specified Result source · Benchmark source