Examenos

Whether an agent calls a tool, asks for missing information or declines, across 3,423 evaluations.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: 3423

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Correct declines in the When2Call action table.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Clarification requests in the When2Call action table.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Correct tool calls in the When2Call action table.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Own run of the public 0.2.1 suite, Overall; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Own run of the public 0.2.1 suite, Knowledge & Reasoning; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Own run of the public 0.2.1 suite, Language Understanding; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Own run of the public 0.2.1 suite, Retrieval & Classification; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Own run of the public 0.2.1 suite, Tools & Automation; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Own run of the public 0.2.1 suite, Arts & Human Taste; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

When2Call (multiple choice) in the Tools & Automation area of the 0.2.1 suite; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Home appliance simulator in the Tools & Automation area of the 0.2.1 suite; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

ToolRet in the Tools & Automation area of the 0.2.1 suite; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

BFCL in the Tools & Automation area of the 0.2.1 suite; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

API-Bank in the Tools & Automation area of the 0.2.1 suite; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Accuracy on Humanity's Last Exam in a single pass.

SubQ Snap 1.1 against Jev 1.14, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Single forward pass on held-out expert exam questions.

SubQ Snap 1.1, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Calibration on probability games from unseen families, published on a 0–1 scale; lower is better.

SubQ Snap 1.1, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Agreement with the Bayes-optimal decision on those games.

SubQ Snap 1.1, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

The vendor's proxy for AA's non-hallucination measure, not an AA result.

SubQ Snap 1.1, vendor run.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Confident errors among 600 unanswerable queries; lower is better.

SubQ Snap 1.1, vendor run.

57 of 600.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: 600

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Long-context multi-document result on the vendor home page.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Nimble 13 accuracy on the vendor home page.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Average of 16 enterprise benchmarks, a subset of HF Decision Index 0.2.1.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

RAGTruth in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

ContractNLI in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

HoVer in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

CRUXEval in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

BANKING77 in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

ACOS in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

FinEntity in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Amazon ESCI in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

CLINC150+OOS in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

NLI4CT in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

BRIGHT in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

PhishNChips in the enterprise subset of HF Decision Index 0.2.1; chance-corrected.

SubQ Snap 1.1 against Jev 1.14, vendor chart on the home page.

Printed in the vendor's own table.

Subquadratic's own runs of each benchmark, not a leaderboard result. HF Decision Index 0.2.1 rows are chance-corrected. The vendor compares Jev 1.14 only; Snap 1.0 is unregistered and its column is not stored. Not ranked here.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-22 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified

Four equally weighted evaluations: AA-Omniscience-Decision, HLE-MultiChoice (513 text-only questions), When2Call and WildGuardTest.

Named SubQ Snap 1.1 by Subquadratic. AA Decisions v1.0. Exact execution dates, precision and serving weight SHA are unpublished.

Manual browser observation of a source-rounded chart label on 2 November 2026; no automated AA adapter.

Independent AA results. Omniscience splits its 25% between accuracy and non-hallucination. Displayed integer chart values, not undisclosed precise values. No automated AA import or refresh adapter.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-17 · Revision: Manually checked chart · 2026-10-17; 2026-11-01; 2026-11-02 · Published observations; result reuse terms not specified

AA Decisions v1 · AA-Omniscience-Decision accuracy

Named SubQ Snap 1.1 by Subquadratic. AA Decisions v1.0. Exact execution dates, precision and serving weight SHA are unpublished.

Manual browser observation of a source-rounded chart label on 2 November 2026; no automated AA adapter.

Source-rounded chart label manually verified, not hidden precise data. Five capability components feed the four-benchmark index. Task cost is average workload cost at source current prices, not a token tariff. Time is median across tasks, not a controlled provider benchmark.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-20 · Revision: Manual rendered chart labels · 2026-10-20; 2026-11-01; 2026-11-02 · Published summary observations; upstream task/result terms retained

AA Decisions v1 · AA-Omniscience-Decision non-hallucination

Named SubQ Snap 1.1 by Subquadratic. AA Decisions v1.0. Exact execution dates, precision and serving weight SHA are unpublished.

Manual browser observation of a source-rounded chart label on 2 November 2026; no automated AA adapter.

Source-rounded chart label manually verified, not hidden precise data. Five capability components feed the four-benchmark index. Task cost is average workload cost at source current prices, not a token tariff. Time is median across tasks, not a controlled provider benchmark.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-20 · Revision: Manual rendered chart labels · 2026-10-20; 2026-11-01; 2026-11-02 · Published summary observations; upstream task/result terms retained

AA Decisions v1 · HLE-MultiChoice

Named SubQ Snap 1.1 by Subquadratic. AA Decisions v1.0. Exact execution dates, precision and serving weight SHA are unpublished.

Manual browser observation of a source-rounded chart label on 2 November 2026; no automated AA adapter.

Source-rounded chart label manually verified, not hidden precise data. Five capability components feed the four-benchmark index. Task cost is average workload cost at source current prices, not a token tariff. Time is median across tasks, not a controlled provider benchmark.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-20 · Revision: Manual rendered chart labels · 2026-10-20; 2026-11-01; 2026-11-02 · Published summary observations; upstream task/result terms retained

AA Decisions v1 · When2Call

Named SubQ Snap 1.1 by Subquadratic. AA Decisions v1.0. Exact execution dates, precision and serving weight SHA are unpublished.

Manual browser observation of a source-rounded chart label on 2 November 2026; no automated AA adapter.

Source-rounded chart label manually verified, not hidden precise data. Five capability components feed the four-benchmark index. Task cost is average workload cost at source current prices, not a token tariff. Time is median across tasks, not a controlled provider benchmark.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-20 · Revision: Manual rendered chart labels · 2026-10-20; 2026-11-01; 2026-11-02 · Published summary observations; upstream task/result terms retained

AA Decisions v1 · WildGuardTest average F1

Named SubQ Snap 1.1 by Subquadratic. AA Decisions v1.0. Exact execution dates, precision and serving weight SHA are unpublished.

Manual browser observation of a source-rounded chart label on 2 November 2026; no automated AA adapter.

Source-rounded chart label manually verified, not hidden precise data. Five capability components feed the four-benchmark index. Task cost is average workload cost at source current prices, not a token tariff. Time is median across tasks, not a controlled provider benchmark.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-20 · Revision: Manual rendered chart labels · 2026-10-20; 2026-11-01; 2026-11-02 · Published summary observations; upstream task/result terms retained

AA Decisions v1 · Median time per task

Named SubQ Snap 1.1 by Subquadratic. AA Decisions v1.0. Exact execution dates, precision and serving weight SHA are unpublished.

Manual browser observation of a source-rounded chart label on 2 November 2026; no automated AA adapter.

Source-rounded chart label manually verified, not hidden precise data. Five capability components feed the four-benchmark index. Task cost is average workload cost at source current prices, not a token tariff. Time is median across tasks, not a controlled provider benchmark.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-20 · Revision: Manual rendered chart labels · 2026-10-20; 2026-11-01; 2026-11-02 · Published summary observations; upstream task/result terms retained

AA Decisions v1 · Cost per million tasks

Named SubQ Snap 1.1 by Subquadratic. AA Decisions v1.0. Exact execution dates, precision and serving weight SHA are unpublished.

Manual browser observation of a source-rounded chart label on 2 November 2026; no automated AA adapter.

Source-rounded chart label manually verified, not hidden precise data. Five capability components feed the four-benchmark index. Task cost is average workload cost at source current prices, not a token tariff. Time is median across tasks, not a controlled provider benchmark.

Checked / reported: 2026-11-02 · Samples: Not published

Snapshot: 2026-10-20 · Revision: Manual rendered chart labels · 2026-10-20; 2026-11-01; 2026-11-02 · Published summary observations; upstream task/result terms retained

Capability Score on the rendered board.

SubQ Snap 1.1 (SubQ) · Closed API (weights not public). Board key subq-snap-1.1; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-22; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://subq.ai.

Rendered board row.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-02 · Evaluated: 2026-10-22 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

Intelligence on the rendered board.

SubQ Snap 1.1 (SubQ) · Closed API (weights not public). Board key subq-snap-1.1; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-22; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://subq.ai.

Rendered board row.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-02 · Evaluated: 2026-10-22 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

Calibration on the rendered board.

SubQ Snap 1.1 (SubQ) · Closed API (weights not public). Board key subq-snap-1.1; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-22; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://subq.ai.

Rendered board row.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-02 · Evaluated: 2026-10-22 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

Cost Score on the rendered board.

SubQ Snap 1.1 (SubQ) · Closed API (weights not public). Board key subq-snap-1.1; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-22; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://subq.ai.

Rendered board row.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-02 · Evaluated: 2026-10-22 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

Modelled cost per 1,000 decisions on the rendered board.

SubQ Snap 1.1 (SubQ) · Closed API (weights not public). Board key subq-snap-1.1; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-22; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://subq.ai.

Rendered board row.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-02 · Evaluated: 2026-10-22 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

API latency, end to end on the rendered board.

SubQ Snap 1.1 (SubQ) · Closed API (weights not public). Board key subq-snap-1.1; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-22; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://subq.ai.

Rendered board row.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-02 · Evaluated: 2026-10-22 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained