Examenos

Typed decisions test split, 2,000 decisions from four workflows, zero-shot.

Amos 1 medium, card c7f72b6. Typed decisions test split, zero-shot.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: 2000

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

Typed decisions test split, 2,000 decisions from four workflows, zero-shot.

Amos 1 medium, card c7f72b6. Typed decisions test split, zero-shot.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: 2000

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

Typed decisions test split, 2,000 decisions from four workflows, zero-shot.

Amos 1 medium, card c7f72b6. Typed decisions test split, zero-shot.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: 2000

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

Per-question priors fitted on the benchmark’s train split and applied at answer time; the weights do not change. Not zero-shot.

Amos 1 medium, card c7f72b6. Priors fitted on the train split; not zero-shot.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: 2000

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

Decision Bench, all 1,071 real records, images included, with the benchmark’s own harness.

Amos 1 medium, card c7f72b6. All 1,071 records with the benchmark’s harness.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: 1071

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

Decision Index 0.3 public part, scored with the board’s own kit; the board’s ranking adds private tests.

Amos 1 medium, card c7f72b6. Public part with the board’s kit.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: Not published

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

JevBench public easy items, as served.

Amos 1 medium, card c7f72b6. As served.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: Not published

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

JevBench public standard items, as served.

Amos 1 medium, card c7f72b6. As served.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: Not published

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

JevBench public hard items, as served.

Amos 1 medium, card c7f72b6. As served.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: Not published

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

Mean over RAGTruth, JudgeBench, Belebele, WinoGrande, ContractNLI, TruthfulQA, Circa, Financial PhraseBank, TabFact and BBH, as served.

Amos 1 medium, card c7f72b6. As served.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: Not published

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

AG News, as served.

Amos 1 medium, card c7f72b6. As served.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: Not published

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

DAIR Emotion, as served.

Amos 1 medium, card c7f72b6. As served.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: Not published

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

Banking77, as served.

Amos 1 medium, card c7f72b6. As served. 77 labels.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: Not published

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

Median server time for a three-question request; hardware not stated.

Amos 1 medium, card c7f72b6. Median, three-question request; hardware not stated.

Printed in the vendor’s own card table.

Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card.

Checked / reported: 2026-11-08 · Samples: Not published

Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified

Capability Score on the rendered board.

Amos 1 medium (Scira) · Open weights · LLM decoder. Board key amos-1-medium; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-medium.

Rendered board row. The row links the Amos 1 medium card and names no commit; it was measured on 13 October, the day the weights were uploaded.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

Intelligence on the rendered board.

Amos 1 medium (Scira) · Open weights · LLM decoder. Board key amos-1-medium; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-medium.

Rendered board row. The row links the Amos 1 medium card and names no commit; it was measured on 13 October, the day the weights were uploaded.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

Calibration on the rendered board.

Amos 1 medium (Scira) · Open weights · LLM decoder. Board key amos-1-medium; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-medium.

Rendered board row. The row links the Amos 1 medium card and names no commit; it was measured on 13 October, the day the weights were uploaded.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

Cost Score on the rendered board.

Amos 1 medium (Scira) · Open weights · LLM decoder. Board key amos-1-medium; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-medium.

Rendered board row. The row links the Amos 1 medium card and names no commit; it was measured on 13 October, the day the weights were uploaded.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

Modelled cost per 1,000 decisions on the rendered board.

Amos 1 medium (Scira) · Open weights · LLM decoder. Board key amos-1-medium; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-medium.

Rendered board row. The row links the Amos 1 medium card and names no commit; it was measured on 13 October, the day the weights were uploaded.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

API latency, end to end on the rendered board.

Amos 1 medium (Scira) · Open weights · LLM decoder. Board key amos-1-medium; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-medium.

Rendered board row. The row links the Amos 1 medium card and names no commit; it was measured on 13 October, the day the weights were uploaded.

Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation.

Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published

Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained

Accuracy for a creator-reported general zero-shot row on the Typed Decisions board.

Exact Amos 1 medium row, source label general, zero-shot, open weights; latency on an H100; the row names no revision.

Creator-reported board row.

Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs.

Checked / reported: 2026-11-08 · Samples: 2000

Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations

KL from gold for a creator-reported general zero-shot row on the Typed Decisions board.

Exact Amos 1 medium row, source label general, zero-shot, open weights; latency on an H100; the row names no revision.

Creator-reported board row.

Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs.

Checked / reported: 2026-11-08 · Samples: 2000

Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations

Brier for a creator-reported general zero-shot row on the Typed Decisions board.

Exact Amos 1 medium row, source label general, zero-shot, open weights; latency on an H100; the row names no revision.

Creator-reported board row.

Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs.

Checked / reported: 2026-11-08 · Samples: 2000

Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations

ECE for a creator-reported general zero-shot row on the Typed Decisions board.

Exact Amos 1 medium row, source label general, zero-shot, open weights; latency on an H100; the row names no revision.

Creator-reported board row.

Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs.

Checked / reported: 2026-11-08 · Samples: 2000

Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations

Median latency for a creator-reported general zero-shot row on the Typed Decisions board.

Exact Amos 1 medium row, source label general, zero-shot, open weights; latency on an H100; the row names no revision.

Creator-reported board row.

Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs.

Checked / reported: 2026-11-08 · Samples: 2000

Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations