Examenos

Four equally weighted evaluations: AA-Omniscience-Decision, HLE-MultiChoice (513 text-only questions), When2Call and WildGuardTest.

Exact named AA decision system; index v1.0. Served precision and detailed execution date not published in inspected chart.

Rounded chart label manually verified in source; not the ordinary language-model Intelligence Index.

Independent AA results. Omniscience splits its 25% between accuracy and non-hallucination. Displayed integer chart values, not undisclosed precise values. No automated AA import or refresh adapter.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-17 · Revision: Manually checked chart · 2026-10-17 · Published observations; result reuse terms not specified

Liquid vendor’s 112 unique evaluations grouped as Average; not an independent index.

Liquid vendor-run October 15 panel. Exact d1-ultra/Jev 1.14/d3 labels; remaining endpoint revisions and served precision unspecified.

Figure-only printed quantity. Source does not publish per-task weights, complete item populations or all inference settings. Average is the vendor’s published aggregate, never recomputed from rounded domains. Peer model exact hosted revisions can be unspecified.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figures 2026-10-15 · Published observations; result reuse terms not specified

Liquid vendor’s 112 unique evaluations grouped as Business and new fields; not an independent index.

Liquid vendor-run October 15 panel. Exact d1-ultra/Jev 1.14/d3 labels; remaining endpoint revisions and served precision unspecified.

Figure-only printed quantity. Source does not publish per-task weights, complete item populations or all inference settings. Average is the vendor’s published aggregate, never recomputed from rounded domains. Peer model exact hosted revisions can be unspecified.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figures 2026-10-15 · Published observations; result reuse terms not specified

Liquid vendor’s 112 unique evaluations grouped as Judgment and taste; not an independent index.

Liquid vendor-run October 15 panel. Exact d1-ultra/Jev 1.14/d3 labels; remaining endpoint revisions and served precision unspecified.

Figure-only printed quantity. Source does not publish per-task weights, complete item populations or all inference settings. Average is the vendor’s published aggregate, never recomputed from rounded domains. Peer model exact hosted revisions can be unspecified.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figures 2026-10-15 · Published observations; result reuse terms not specified

Liquid vendor’s 112 unique evaluations grouped as Knowledge and reasoning; not an independent index.

Liquid vendor-run October 15 panel. Exact d1-ultra/Jev 1.14/d3 labels; remaining endpoint revisions and served precision unspecified.

Figure-only printed quantity. Source does not publish per-task weights, complete item populations or all inference settings. Average is the vendor’s published aggregate, never recomputed from rounded domains. Peer model exact hosted revisions can be unspecified.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figures 2026-10-15 · Published observations; result reuse terms not specified

Liquid vendor’s 112 unique evaluations grouped as Languages; not an independent index.

Liquid vendor-run October 15 panel. Exact d1-ultra/Jev 1.14/d3 labels; remaining endpoint revisions and served precision unspecified.

Figure-only printed quantity. Source does not publish per-task weights, complete item populations or all inference settings. Average is the vendor’s published aggregate, never recomputed from rounded domains. Peer model exact hosted revisions can be unspecified.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figures 2026-10-15 · Published observations; result reuse terms not specified

Liquid vendor’s 112 unique evaluations grouped as Pictures; not an independent index.

Liquid vendor-run October 15 panel. Exact d1-ultra/Jev 1.14/d3 labels; remaining endpoint revisions and served precision unspecified.

Figure-only printed quantity. Source does not publish per-task weights, complete item populations or all inference settings. Average is the vendor’s published aggregate, never recomputed from rounded domains. Peer model exact hosted revisions can be unspecified.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figures 2026-10-15 · Published observations; result reuse terms not specified

Liquid vendor’s 112 unique evaluations grouped as Robustness; not an independent index.

Liquid vendor-run October 15 panel. Exact d1-ultra/Jev 1.14/d3 labels; remaining endpoint revisions and served precision unspecified.

Figure-only printed quantity. Source does not publish per-task weights, complete item populations or all inference settings. Average is the vendor’s published aggregate, never recomputed from rounded domains. Peer model exact hosted revisions can be unspecified.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figures 2026-10-15 · Published observations; result reuse terms not specified

Liquid vendor’s 112 unique evaluations grouped as Safety and policy; not an independent index.

Liquid vendor-run October 15 panel. Exact d1-ultra/Jev 1.14/d3 labels; remaining endpoint revisions and served precision unspecified.

Figure-only printed quantity. Source does not publish per-task weights, complete item populations or all inference settings. Average is the vendor’s published aggregate, never recomputed from rounded domains. Peer model exact hosted revisions can be unspecified.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figures 2026-10-15 · Published observations; result reuse terms not specified

Liquid vendor’s 112 unique evaluations grouped as Tools and agents; not an independent index.

Liquid vendor-run October 15 panel. Exact d1-ultra/Jev 1.14/d3 labels; remaining endpoint revisions and served precision unspecified.

Figure-only printed quantity. Source does not publish per-task weights, complete item populations or all inference settings. Average is the vendor’s published aggregate, never recomputed from rounded domains. Peer model exact hosted revisions can be unspecified.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figures 2026-10-15 · Published observations; result reuse terms not specified

Vendor-defined mean of absolute accuracy minus chosen confidence; not interchangeable with other studies’ ECE.

Liquid vendor figure: mean absolute accuracy−confidence on chosen option; hardware/precision unpublished.

Printed figure values manually inspected. Curve coordinates are not reconstructed. Source task weighting and binning are not fully published.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-15 · Revision: Release figure 2026-10-15 · Published observations; result reuse terms not specified

Exact named hosted API observation; All-case accuracy.

Named source route liquid/d1-ultra; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 56

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Hard-case accuracy.

Named source route liquid/d1-ultra; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 19

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Public-case accuracy.

Named source route liquid/d1-ultra; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 14

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Hidden-case accuracy.

Named source route liquid/d1-ultra; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 42

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Brier-based confidence score.

Named source route liquid/d1-ultra; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 56

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified