joey-1 Decision Model Results

joey-1

Explore joey-1: exact decision configuration, published specifications, source-linked benchmarks and API tariffs when available.

Published specifications

Provider: Baseten
Version: baselabs/joey-1 (2B)
System type: Trained checkpoint
Access: Open weights
License: MIT
Inputs: text, image
Decision types: Choice, Yes / no, Score
Published size: 2.72B
Active parameters: Not published
Context / input limit: 16,384 tokens (max sequence)
Announced: 2026-10-29

Hugging Face · Announcement · Website · OpenRouter

Model notes

Base Labs is the research team at Baseten. Fine-tune of Qwen3.5-2B on the 6.21M-row Mix 8 with the vision encoder frozen; MIT license. Max sequence 16,384 tokens. Baseten's API name `decision-models/baseten-joey-2b-latest` is a moving alias, so results through it are not pinned to a commit. OpenRouter endpoint label baseten/joey-1-20261029. Joey-1 launched 29 October 2026, after the free week. Typed outputs are probabilities over supplied options; confidence is not a correctness guarantee.

Checkpoint revision: 9efbe485efbed9598dfca5294cbbe8da00e21908

← All decision models · Compare this decision system →

Published API tariff

$0.01 per 1M input tokens

Baseten · checked 2026-11-02

Baseten library page: output and cached input show $0.00. Free for a launch week, which ended 27 October 2026.

No output-token charge does not mean free requests. These are provider tariffs, separate from benchmark workload costs.

336 recorded observations. Published results retain their source and setup; no Examenos evaluation was run. Missing results are not zero. How to read these comparisons →

Official / vendor · Base Labs model cards (vendor reported)

joey-1 · Official / vendor · Base Labs model cards (vendor reported)
Evaluation	Result	Evidence
Base Labs cards · Jev Decision Index 0.4 public split	63.30	Setup & source Public split as printed in the joey-1 card, #37 of 230 for joey-1 when written. Printed in the joey-1 card. Printed in the vendor's own table. Base Labs' own figures from the model cards and report. Its Decision Index values are from the 0.4 board when the report was written, so they differ from the community 0.4.1 rows here. Its Artificial Analysis replica is a vendor rerun and is never an AA row. Not ranked here. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-21 · Revision: mammoth 61cec44c28 elephant b3f8a93462 joey 9efbe485ef · Published observations; result reuse terms not specified Result source · Benchmark source
Base Labs cards · Replica of the Artificial Analysis Decision Index	62.9	Setup & source Base Labs' own replica of AA's index. AA's published values are stored separately. Base Labs' own replica; not an Artificial Analysis evaluation. Printed in the vendor's own table. Base Labs' own figures from the model cards and report. Its Decision Index values are from the 0.4 board when the report was written, so they differ from the community 0.4.1 rows here. Its Artificial Analysis replica is a vendor rerun and is never an AA row. Not ranked here. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-21 · Revision: mammoth 61cec44c28 elephant b3f8a93462 joey 9efbe485ef · Published observations; result reuse terms not specified Result source · Benchmark source

Official / vendor · Baseten Joey-1 launch post (vendor measured)

joey-1 · Official / vendor · Baseten Joey-1 launch post (vendor measured)
Evaluation	Result	Evidence
Baseten · P50 end-to-end latency, average 2,000-token request	20 ms	Setup & source Baseten production-scale P50 for the average 2,000-token request. Baseten Model APIs at production scale; average 2,000-token request. Printed in the vendor's own table. Baseten production-scale latency on its own service, not a controlled comparison. Not ranked here. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-29 · Revision: Read 2026-11-02 · Published observations; result reuse terms not specified Result source · Benchmark source

Evaluator-published · Community Decision Index

Independent · SafeAgents Judge Bench

joey-1 · Independent · SafeAgents Judge Bench
Evaluation	Result	Evidence
SafeAgents Judge Bench v0 · Balanced accuracy	50.2%	Setup & source Block-or-allow balanced accuracy from the noul question. Exact Hugging Face checkpoint baselabs/joey-1; hardware NVIDIA L4 (HF Jobs); bench revision 5671a11198785784d72822b8df1bce2238f726a4. Model revision and precision not stated on the card. Published as a fraction; raw value retained. Real agent tool calls that a scanner marked suspicious: 562 calls (523 block, 39 allow) from 91 trajectories over 47 tasks, all in test. One System One request per call with a noul question (block?) and a 7-way category choice; block at P(block) ≥ 0.5. Caveats from the card: few allowed calls make the false-block rate noisy, one agent dominates the traces, question wording matters, and labels come from a four-model panel. The card records 16 calls decided by a human where the panel split, and also says the labels are not yet audited by a human and 2-2 splits are excluded; both statements are kept here as published. Checked / reported: 2026-11-02 · Samples: 562 Snapshot: 2026-10-30 · Revision: 5671a11198785784d72822b8df1bce2238f726a4 · Apache-2.0 dataset card; published observations Result source · Benchmark source
SafeAgents Judge Bench v0 · AUROC	48.2%	Setup & source AUROC on P(block). Exact Hugging Face checkpoint baselabs/joey-1; hardware NVIDIA L4 (HF Jobs); bench revision 5671a11198785784d72822b8df1bce2238f726a4. Model revision and precision not stated on the card. Published as a fraction; raw value retained. Real agent tool calls that a scanner marked suspicious: 562 calls (523 block, 39 allow) from 91 trajectories over 47 tasks, all in test. One System One request per call with a noul question (block?) and a 7-way category choice; block at P(block) ≥ 0.5. Caveats from the card: few allowed calls make the false-block rate noisy, one agent dominates the traces, question wording matters, and labels come from a four-model panel. The card records 16 calls decided by a human where the panel split, and also says the labels are not yet audited by a human and 2-2 splits are excluded; both statements are kept here as published. Checked / reported: 2026-11-02 · Samples: 562 Snapshot: 2026-10-30 · Revision: 5671a11198785784d72822b8df1bce2238f726a4 · Apache-2.0 dataset card; published observations Result source · Benchmark source
SafeAgents Judge Bench v0 · Block recall	15.9%	Setup & source Share of block calls blocked. Exact Hugging Face checkpoint baselabs/joey-1; hardware NVIDIA L4 (HF Jobs); bench revision 5671a11198785784d72822b8df1bce2238f726a4. Model revision and precision not stated on the card. Published as a fraction; raw value retained. Real agent tool calls that a scanner marked suspicious: 562 calls (523 block, 39 allow) from 91 trajectories over 47 tasks, all in test. One System One request per call with a noul question (block?) and a 7-way category choice; block at P(block) ≥ 0.5. Caveats from the card: few allowed calls make the false-block rate noisy, one agent dominates the traces, question wording matters, and labels come from a four-model panel. The card records 16 calls decided by a human where the panel split, and also says the labels are not yet audited by a human and 2-2 splits are excluded; both statements are kept here as published. Checked / reported: 2026-11-02 · Samples: 523 Snapshot: 2026-10-30 · Revision: 5671a11198785784d72822b8df1bce2238f726a4 · Apache-2.0 dataset card; published observations Result source · Benchmark source
SafeAgents Judge Bench v0 · False blocks	15.4%	Setup & source Share of the 39 allowed calls blocked. Exact Hugging Face checkpoint baselabs/joey-1; hardware NVIDIA L4 (HF Jobs); bench revision 5671a11198785784d72822b8df1bce2238f726a4. Model revision and precision not stated on the card. Published as a fraction; raw value retained. Real agent tool calls that a scanner marked suspicious: 562 calls (523 block, 39 allow) from 91 trajectories over 47 tasks, all in test. One System One request per call with a noul question (block?) and a 7-way category choice; block at P(block) ≥ 0.5. Caveats from the card: few allowed calls make the false-block rate noisy, one agent dominates the traces, question wording matters, and labels come from a four-model panel. The card records 16 calls decided by a human where the panel split, and also says the labels are not yet audited by a human and 2-2 splits are excluded; both statements are kept here as published. Checked / reported: 2026-11-02 · Samples: 39 Snapshot: 2026-10-30 · Revision: 5671a11198785784d72822b8df1bce2238f726a4 · Apache-2.0 dataset card; published observations Result source · Benchmark source
SafeAgents Judge Bench v0 · Category macro F1	21.9%	Setup & source Macro F1 on the 7-way category question. Exact Hugging Face checkpoint baselabs/joey-1; hardware NVIDIA L4 (HF Jobs); bench revision 5671a11198785784d72822b8df1bce2238f726a4. Model revision and precision not stated on the card. Published as a fraction; raw value retained. Real agent tool calls that a scanner marked suspicious: 562 calls (523 block, 39 allow) from 91 trajectories over 47 tasks, all in test. One System One request per call with a noul question (block?) and a 7-way category choice; block at P(block) ≥ 0.5. Caveats from the card: few allowed calls make the false-block rate noisy, one agent dominates the traces, question wording matters, and labels come from a four-model panel. The card records 16 calls decided by a human where the panel split, and also says the labels are not yet audited by a human and 2-2 splits are excluded; both statements are kept here as published. Checked / reported: 2026-11-02 · Samples: 562 Snapshot: 2026-10-30 · Revision: 5671a11198785784d72822b8df1bce2238f726a4 · Apache-2.0 dataset card; published observations Result source · Benchmark source
SafeAgents Judge Bench v0 · Category accuracy	39.5%	Setup & source Accuracy on the 7-way category question. Exact Hugging Face checkpoint baselabs/joey-1; hardware NVIDIA L4 (HF Jobs); bench revision 5671a11198785784d72822b8df1bce2238f726a4. Model revision and precision not stated on the card. Published as a fraction; raw value retained. Real agent tool calls that a scanner marked suspicious: 562 calls (523 block, 39 allow) from 91 trajectories over 47 tasks, all in test. One System One request per call with a noul question (block?) and a 7-way category choice; block at P(block) ≥ 0.5. Caveats from the card: few allowed calls make the false-block rate noisy, one agent dominates the traces, question wording matters, and labels come from a four-model panel. The card records 16 calls decided by a human where the panel split, and also says the labels are not yet audited by a human and 2-2 splits are excluded; both statements are kept here as published. Checked / reported: 2026-11-02 · Samples: 562 Snapshot: 2026-10-30 · Revision: 5671a11198785784d72822b8df1bce2238f726a4 · Apache-2.0 dataset card; published observations Result source · Benchmark source
SafeAgents Judge Bench v0 · Balanced accuracy via category	58.2%	Setup & source Block-or-allow balanced accuracy read from the category question. Exact Hugging Face checkpoint baselabs/joey-1; hardware NVIDIA L4 (HF Jobs); bench revision 5671a11198785784d72822b8df1bce2238f726a4. Model revision and precision not stated on the card. Published as a fraction; raw value retained. Real agent tool calls that a scanner marked suspicious: 562 calls (523 block, 39 allow) from 91 trajectories over 47 tasks, all in test. One System One request per call with a noul question (block?) and a 7-way category choice; block at P(block) ≥ 0.5. Caveats from the card: few allowed calls make the false-block rate noisy, one agent dominates the traces, question wording matters, and labels come from a four-model panel. The card records 16 calls decided by a human where the panel split, and also says the labels are not yet audited by a human and 2-2 splits are excluded; both statements are kept here as published. Checked / reported: 2026-11-02 · Samples: 562 Snapshot: 2026-10-30 · Revision: 5671a11198785784d72822b8df1bce2238f726a4 · Apache-2.0 dataset card; published observations Result source · Benchmark source
SafeAgents Judge Bench v0 · Misbehaving trajectories caught	23 count	Setup & source Of 78 misbehaving trajectories, those with at least one bad call blocked. Exact Hugging Face checkpoint baselabs/joey-1; hardware NVIDIA L4 (HF Jobs); bench revision 5671a11198785784d72822b8df1bce2238f726a4. Model revision and precision not stated on the card. 23 of 78 misbehaving trajectories. Real agent tool calls that a scanner marked suspicious: 562 calls (523 block, 39 allow) from 91 trajectories over 47 tasks, all in test. One System One request per call with a noul question (block?) and a 7-way category choice; block at P(block) ≥ 0.5. Caveats from the card: few allowed calls make the false-block rate noisy, one agent dominates the traces, question wording matters, and labels come from a four-model panel. The card records 16 calls decided by a human where the panel split, and also says the labels are not yet audited by a human and 2-2 splits are excluded; both statements are kept here as published. Checked / reported: 2026-11-02 · Samples: 78 Snapshot: 2026-10-30 · Revision: 5671a11198785784d72822b8df1bce2238f726a4 · Apache-2.0 dataset card; published observations Result source · Benchmark source
SafeAgents Judge Bench v0 · P50 latency	0.118 s	Setup & source Per decision, one request at a time, on the hardware shown. Exact Hugging Face checkpoint baselabs/joey-1; hardware NVIDIA L4 (HF Jobs); bench revision 5671a11198785784d72822b8df1bce2238f726a4. Model revision and precision not stated on the card. Mac and GPU numbers are not comparable, so this is not ranked. Real agent tool calls that a scanner marked suspicious: 562 calls (523 block, 39 allow) from 91 trajectories over 47 tasks, all in test. One System One request per call with a noul question (block?) and a 7-way category choice; block at P(block) ≥ 0.5. Caveats from the card: few allowed calls make the false-block rate noisy, one agent dominates the traces, question wording matters, and labels come from a four-model panel. The card records 16 calls decided by a human where the panel split, and also says the labels are not yet audited by a human and 2-2 splits are excluded; both statements are kept here as published. Checked / reported: 2026-11-02 · Samples: 562 Snapshot: 2026-10-30 · Revision: 5671a11198785784d72822b8df1bce2238f726a4 · Apache-2.0 dataset card; published observations Result source · Benchmark source

Independent · ModelTested

joey-1 · Independent · ModelTested
Evaluation	Result	Evidence
ModelTested 56-case · All-case accuracy	80.40%	Setup & source Exact named hosted API observation; All-case accuracy. Named source route baseten/joey-1; ModelTested runner v1; tested_on=openrouter. Backend weight revision and API inference precision unpublished. Publisher generated 2026-11-01; board added this route 2026-10-30. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported. 56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0. Rows added 1 November 2026 use revision 668a52ad3b932ca910fd985f96dcdf0a85d2dae1, where the earlier 11 systems are unchanged. Rows added 2 November 2026 use revision 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0. Checked / reported: 2026-11-01 · Samples: 56 Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23; 668a52ad3b932ca910fd985f96dcdf0a85d2dae1 for Drex Max, Jev 1.14 and d1.1; 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0 for Mammoth, Elephant, Joey, Span-02 and both Solar Decide routes · Published aggregate observations; result reuse terms not separately specified Result source · Benchmark source
ModelTested 56-case · Hard-case accuracy	65.80%	Setup & source Exact named hosted API observation; Hard-case accuracy. Named source route baseten/joey-1; ModelTested runner v1; tested_on=openrouter. Backend weight revision and API inference precision unpublished. Publisher generated 2026-11-01; board added this route 2026-10-30. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported. 56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0. Rows added 1 November 2026 use revision 668a52ad3b932ca910fd985f96dcdf0a85d2dae1, where the earlier 11 systems are unchanged. Rows added 2 November 2026 use revision 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0. Checked / reported: 2026-11-01 · Samples: 56 Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23; 668a52ad3b932ca910fd985f96dcdf0a85d2dae1 for Drex Max, Jev 1.14 and d1.1; 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0 for Mammoth, Elephant, Joey, Span-02 and both Solar Decide routes · Published aggregate observations; result reuse terms not separately specified Result source · Benchmark source
ModelTested 56-case · Public-case accuracy	85.70%	Setup & source Exact named hosted API observation; Public-case accuracy. Named source route baseten/joey-1; ModelTested runner v1; tested_on=openrouter. Backend weight revision and API inference precision unpublished. Publisher generated 2026-11-01; board added this route 2026-10-30. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported. 56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0. Rows added 1 November 2026 use revision 668a52ad3b932ca910fd985f96dcdf0a85d2dae1, where the earlier 11 systems are unchanged. Rows added 2 November 2026 use revision 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0. Checked / reported: 2026-11-01 · Samples: 56 Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23; 668a52ad3b932ca910fd985f96dcdf0a85d2dae1 for Drex Max, Jev 1.14 and d1.1; 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0 for Mammoth, Elephant, Joey, Span-02 and both Solar Decide routes · Published aggregate observations; result reuse terms not separately specified Result source · Benchmark source
ModelTested 56-case · Hidden-case accuracy	78.60%	Setup & source Exact named hosted API observation; Hidden-case accuracy. Named source route baseten/joey-1; ModelTested runner v1; tested_on=openrouter. Backend weight revision and API inference precision unpublished. Publisher generated 2026-11-01; board added this route 2026-10-30. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported. 56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0. Rows added 1 November 2026 use revision 668a52ad3b932ca910fd985f96dcdf0a85d2dae1, where the earlier 11 systems are unchanged. Rows added 2 November 2026 use revision 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0. Checked / reported: 2026-11-01 · Samples: 56 Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23; 668a52ad3b932ca910fd985f96dcdf0a85d2dae1 for Drex Max, Jev 1.14 and d1.1; 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0 for Mammoth, Elephant, Joey, Span-02 and both Solar Decide routes · Published aggregate observations; result reuse terms not separately specified Result source · Benchmark source
ModelTested 56-case · Brier-based confidence score	86.60	Setup & source Exact named hosted API observation; Brier-based confidence score. Named source route baseten/joey-1; ModelTested runner v1; tested_on=openrouter. Backend weight revision and API inference precision unpublished. Publisher generated 2026-11-01; board added this route 2026-10-30. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported. 56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0. Rows added 1 November 2026 use revision 668a52ad3b932ca910fd985f96dcdf0a85d2dae1, where the earlier 11 systems are unchanged. Rows added 2 November 2026 use revision 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0. Checked / reported: 2026-11-01 · Samples: 56 Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23; 668a52ad3b932ca910fd985f96dcdf0a85d2dae1 for Drex Max, Jev 1.14 and d1.1; 5c6c19bb3b7332ccd28c8c13bd32cdc69cedc8f0 for Mammoth, Elephant, Joey, Span-02 and both Solar Decide routes · Published aggregate observations; result reuse terms not separately specified Result source · Benchmark source

joey-1 · Evaluator-published · Community Decision Index
Evaluation	Result	Evidence
HF text 0.4.1 · Full score	35.91	Setup & source HF text 0.4.1 · Full score Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board does not name a served checkpoint in this repository history. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Public index	63.30	Setup & source HF text 0.4 · Public index Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Private same-skill component	50.16	Setup & source HF text 0.4 · Private same-skill component Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4.1 · Private new-domain component	10.20	Setup & source HF text 0.4.1 · Private new-domain component Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board does not name a served checkpoint in this repository history. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Raw public index	72.47	Setup & source HF text 0.4 · Raw public index Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Public breadth index	61.97	Setup & source HF text 0.4 · Public breadth index Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Equated private same-skill	49.69	Setup & source HF text 0.4 · Equated private same-skill Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4.1 · Equated private new domains	15.25	Setup & source HF text 0.4.1 · Equated private new domains Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board does not name a served checkpoint in this repository history. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Full contribution · public	0.00	Setup & source HF text 0.4 · Full contribution · public Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Full contribution · same_skills	29.81	Setup & source HF text 0.4 · Full contribution · same_skills Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4.1 · Full contribution · new_domains	6.10	Setup & source HF text 0.4.1 · Full contribution · new_domains Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. The board does not name a served checkpoint in this repository history. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Knowledge · raw	56.60	Setup & source HF text 0.4 · Knowledge · raw Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.566, "skill": 0.4509, "combined": 0.2739, "coverage": 1, "pending": 0, "n": 11} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 11 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Knowledge · skill	45.09	Setup & source HF text 0.4 · Knowledge · skill Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.566, "skill": 0.4509, "combined": 0.2739, "coverage": 1, "pending": 0, "n": 11} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 11 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Knowledge · combined	27.39	Setup & source HF text 0.4 · Knowledge · combined Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.566, "skill": 0.4509, "combined": 0.2739, "coverage": 1, "pending": 0, "n": 11} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 11 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Language · raw	74.53	Setup & source HF text 0.4 · Language · raw Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.7453, "skill": 0.6283, "combined": 0.4681, "coverage": 1, "pending": 0, "n": 9} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 9 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Language · skill	62.83	Setup & source HF text 0.4 · Language · skill Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.7453, "skill": 0.6283, "combined": 0.4681, "coverage": 1, "pending": 0, "n": 9} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 9 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Language · combined	46.81	Setup & source HF text 0.4 · Language · combined Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.7453, "skill": 0.6283, "combined": 0.4681, "coverage": 1, "pending": 0, "n": 9} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 9 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Retrieval · raw	77.15	Setup & source HF text 0.4 · Retrieval · raw Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7715, "skill": 0.7115, "combined": 0.548, "coverage": 1, "pending": 0, "n": 6} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Retrieval · skill	71.15	Setup & source HF text 0.4 · Retrieval · skill Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7715, "skill": 0.7115, "combined": 0.548, "coverage": 1, "pending": 0, "n": 6} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Retrieval · combined	54.80	Setup & source HF text 0.4 · Retrieval · combined Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7715, "skill": 0.7115, "combined": 0.548, "coverage": 1, "pending": 0, "n": 6} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Tools · raw	87.60	Setup & source HF text 0.4 · Tools · raw Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.876, "skill": 0.8581, "combined": 0.7512, "coverage": 1, "pending": 0, "n": 5} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 5 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Tools · skill	85.81	Setup & source HF text 0.4 · Tools · skill Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.876, "skill": 0.8581, "combined": 0.7512, "coverage": 1, "pending": 0, "n": 5} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 5 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Tools · combined	75.12	Setup & source HF text 0.4 · Tools · combined Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.876, "skill": 0.8581, "combined": 0.7512, "coverage": 1, "pending": 0, "n": 5} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 5 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Arts · raw	71.16	Setup & source HF text 0.4 · Arts · raw Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7116, "skill": 0.5474, "combined": 0.5189, "coverage": 1, "pending": 0, "n": 6} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Arts · skill	54.74	Setup & source HF text 0.4 · Arts · skill Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7116, "skill": 0.5474, "combined": 0.5189, "coverage": 1, "pending": 0, "n": 6} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Arts · combined	51.89	Setup & source HF text 0.4 · Arts · combined Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7116, "skill": 0.5474, "combined": 0.5189, "coverage": 1, "pending": 0, "n": 6} The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · ECE	4.44 pp	Setup & source HF text calibration 0.4 · ECE Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Brier	0.3139 Brier	Setup & source HF text calibration 0.4 · Brier Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Sample accuracy	76.42%	Setup & source HF text calibration 0.4 · Sample accuracy Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Mean chosen confidence	80.81%	Setup & source HF text calibration 0.4 · Mean chosen confidence Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Wrong among answers at ≥95% confidence	1.53%	Setup & source HF text calibration 0.4 · Wrong among answers at ≥95% confidence Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Knowledge · ECE	5.21 pp	Setup & source HF text calibration 0.4 · Knowledge · ECE Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Knowledge · Sample accuracy	65.12%	Setup & source HF text calibration 0.4 · Knowledge · Sample accuracy Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Knowledge · Mean chosen confidence	69.34%	Setup & source HF text calibration 0.4 · Knowledge · Mean chosen confidence Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Language · ECE	5.59 pp	Setup & source HF text calibration 0.4 · Language · ECE Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Language · Sample accuracy	83.65%	Setup & source HF text calibration 0.4 · Language · Sample accuracy Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Language · Mean chosen confidence	89.24%	Setup & source HF text calibration 0.4 · Language · Mean chosen confidence Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Retrieval · ECE	5.16 pp	Setup & source HF text calibration 0.4 · Retrieval · ECE Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Retrieval · Sample accuracy	80.38%	Setup & source HF text calibration 0.4 · Retrieval · Sample accuracy Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Retrieval · Mean chosen confidence	85.53%	Setup & source HF text calibration 0.4 · Retrieval · Mean chosen confidence Exact source engine joey-1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 2721801024, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/baselabs/joey-1", "code_url": null, "base_model": "Qwen/Qwen3.5-2B", "base_url": "https://huggingface.co/Qwen/Qwen3.5-2B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "baselabs", "name": "baselabs", "logo": "data/orgs/baselabs.png"}, "pr": 330, "new": true, "show_notes": true}. New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Decision Index 0.4.1 (release-v4), source generated 2026-10-29T14:04:16Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #330). Submitted public 0.3 run adopted (kit rescore 63.30, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.60% on 831 rows (bar 98.5%). Same author and serving code as elephant-1 and mammoth-1; the weights repository is gated (manual approval). Training includes train splits of Decision Index benchmarks (author-declared). One private new-domain test is not counted for this model: our checks found signs its training data overlapped it. Its new-domain score uses the remaining tests, put on the same scale with the same stock models. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board does not name a served checkpoint in this repository history. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-02 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source