Nimble-v4-max v4.1 Decision Model Results

Nimble-v4-max v4.1

Explore Nimble-v4-max v4.1: exact decision configuration, published specifications, source-linked benchmarks and API tariffs when available.

Latest versionPrevious version: Nimble-v4-max v4.0Release source ↗

Published specifications

Provider: Bespoke Labs
Version: v4.1 (tag v4.1) · bespokelabs/Nimble-v4-max
System type: Trained checkpoint
Access: Open weights
License: CC BY-NC 4.0
Inputs: text
Decision types: Choice, Yes / no, Score
Published size: 27.36B
Active parameters: Not published
Context / input limit: 32,768 tokens (longer requests are refused, never cut)
Announced: 2026-11-03

Hugging Face · Announcement · Website

Model notes

Version 4.1 of Nimble-v4-max, a fine-tune of Qwen/Qwen3.8-27B: new weights in the same repository (all 11 safetensors shards changed; tag v4.1 is commit a29d86e, 3 November 2026). The card calls it a drop-in replacement for v4.0, with the same requests and inference code; its default calibration temperature is 2.25 (v4.0: 1.5), and temperature never changes which option ranks first. Context per the results dataset: requests over 32,768 tokens are refused as unsupported. About 55 GB in BF16, so one 80 GB or larger GPU. Loading it as a plain chat model does not reproduce its answers. Bespoke's docs, read 5 November 2026, list the hosted nimble-v4-max and nimble-latest at $0.04 per 1M input tokens and do not say whether the hosted model moved to v4.1, so no API name or tariff is attached to this entry. CC BY-NC 4.0; the Qwen base is Apache-2.0.

Checkpoint revision: a29d86e2522ae2f0fbdff2051fdf0554a3134ad0

← All decision models · Compare this decision system →

362 recorded observations. Published results retain their source and setup; no Examenos evaluation was run. Missing results are not zero. How to read these comparisons →

Evaluator-published · Community Decision Index

Official / vendor · Nimble-v4-max model card (vendor measured)

Nimble-v4-max v4.1 · Official / vendor · Nimble-v4-max model card (vendor measured)
Evaluation	Result	Evidence
Nimble-v4-max card · Decision Index 0.3 public suite	72.74	Setup & source Own run of the public 0.3 suite. Own run of release v4.1 (tag v4.1); results in bespokelabs/nimble-v4-max-decision-index, run nimble-v4.1-max, scored 3 November 2026 with edition 0.3. Printed in the card's version table and in the run's scores.json (balanced_skill 72.74). Bespoke Labs' own run of the public 0.3 suite on 12 October 2026, published with a full results dataset. Not ranked here. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-18 · Revision: 655aa7051c2ab39660193ebb70c8926d3d2cb7aa (v4.0); a29d86e2522ae2f0fbdff2051fdf0554a3134ad0 (v4.1) · Published observations; result reuse terms not specified Result source · Benchmark source

Official / vendor · Bespoke Labs results dataset (vendor measured)

Nimble-v4-max v4.1 · Official / vendor · Bespoke Labs results dataset (vendor measured)
Evaluation	Result	Evidence
Nimble-v4-max Decision Index runs · BFCL (case exact accuracy)	98.29%	Setup & source BFCL, case exact accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (1,694 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9828807556080283 on a 0–1 scale (case exact accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 1694 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · ToolRet (nDCG@10)	73.31%	Setup & source ToolRet, nDCG@10, from Bespoke Labs' own run of the Decision Index 0.3 public suite (685 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.7330700535780103 on a 0–1 scale (nDCG@10). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 685 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · API-Bank (accuracy)	88.19%	Setup & source API-Bank, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (508 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8818897637795275 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 508 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · BANKING77 (macro-F1)	93.15%	Setup & source BANKING77, macro-F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (3,080 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9314605889089179 on a 0–1 scale (macro-F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 3080 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · CLINC150+OOS (macro-F1)	96.25%	Setup & source CLINC150+OOS, macro-F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (5,500 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.962513647528099 on a 0–1 scale (macro-F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 5500 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · RouterBench (selected quality (quality objective))	79.49%	Setup & source RouterBench, selected quality (quality objective), from Bespoke Labs' own run of the Decision Index 0.3 public suite (10,000 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.79487 on a 0–1 scale (selected quality (quality objective)). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 10000 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · Home appliance simulator (case exact accuracy)	100.00%	Setup & source Home appliance simulator, case exact accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (88 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 1 on a 0–1 scale (case exact accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 88 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · SGD/SGD-X (macro-F1)	73.64%	Setup & source SGD/SGD-X, macro-F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (2,500 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.7363717971014123 on a 0–1 scale (macro-F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 2500 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · ContractNLI (macro-F1)	87.08%	Setup & source ContractNLI, macro-F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (123 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8708496655205263 on a 0–1 scale (macro-F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 123 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · ANLI (macro-F1)	78.67%	Setup & source ANLI, macro-F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (3,200 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.7867143545861129 on a 0–1 scale (macro-F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 3200 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · BPoMP (accuracy)	99.16%	Setup & source BPoMP, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (5,000 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9916 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 5000 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · Humicroedit (accuracy)	66.97%	Setup & source Humicroedit, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (2,628 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.669710806697108 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 2628 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · POP909-CL (accuracy)	84.75%	Setup & source POP909-CL, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (2,000 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8475 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 2000 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · cfcolor (accuracy)	70.80%	Setup & source cfcolor, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (5,000 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.708 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 5000 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · MMLU (accuracy)	88.29%	Setup & source MMLU, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (14,033 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8829188341765838 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 14033 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · GPQA Diamond (accuracy)	55.61%	Setup & source GPQA Diamond, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (196 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.5561224489795918 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 196 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · ARC-Easy (accuracy)	98.95%	Setup & source ARC-Easy, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (2,376 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9894781144781145 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 2376 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · ARC-Challenge (accuracy)	97.10%	Setup & source ARC-Challenge, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (1,172 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9709897610921502 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 1172 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · WinoGrande (accuracy)	94.55%	Setup & source WinoGrande, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (1,267 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9455406471981057 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 1267 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · HellaSwag (accuracy)	97.67%	Setup & source HellaSwag, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (10,042 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9766978689504083 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 10042 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · GSM8K (accuracy)	83.59%	Setup & source GSM8K, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (2,638 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8358605003790751 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 2638 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · ChessBench (accuracy)	27.14%	Setup & source ChessBench, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (5,000 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.2714 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 5000 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · MuSR (accuracy)	75.66%	Setup & source MuSR, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (752 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.7566489361702128 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 752 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · SATA-Bench (case exact accuracy)	60.24%	Setup & source SATA-Bench, case exact accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (1,650 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.6024242424242424 on a 0–1 scale (case exact accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 1650 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · SimpleBench (accuracy)	20.00%	Setup & source SimpleBench, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (10 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.2 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 10 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · BRIGHT (nDCG@10)	48.61%	Setup & source BRIGHT, nDCG@10, from Bespoke Labs' own run of the Decision Index 0.3 public suite (220 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.48613036902346274 on a 0–1 scale (nDCG@10). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 220 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · Amazon ESCI (macro-F1)	62.12%	Setup & source Amazon ESCI, macro-F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (5,000 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.6212273091372763 on a 0–1 scale (macro-F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 5000 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · ACOS (per-review F1)	66.10%	Setup & source ACOS, per-review F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (1,565 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.6610119047619047 on a 0–1 scale (per-review F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 1565 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · FinEntity (macro-F1)	93.78%	Setup & source FinEntity, macro-F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (979 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9377921871116485 on a 0–1 scale (macro-F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 979 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · VAST (macro-F1)	82.02%	Setup & source VAST, macro-F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (3,006 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8202328555931823 on a 0–1 scale (macro-F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 3006 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · NLI4CT (macro-F1)	88.39%	Setup & source NLI4CT, macro-F1, from Bespoke Labs' own run of the Decision Index 0.3 public suite (5,500 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8838804597701149 on a 0–1 scale (macro-F1). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 5500 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · CRUXEval (accuracy)	86.49%	Setup & source CRUXEval, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (570 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8649122807017544 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 570 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · CLadder (accuracy)	92.94%	Setup & source CLadder, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (5,000 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9294 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 5000 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · HLE (accuracy)	15.17%	Setup & source HLE, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (501 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.15169660678642716 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 501 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · Habermas Machine (accuracy)	62.83%	Setup & source Habermas Machine, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (1,676 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.6282816229116945 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 1676 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · PhishNChips phishing decisions (accuracy)	84.75%	Setup & source PhishNChips phishing decisions, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (2,000 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8475 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 2000 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · MMLU-Pro (accuracy)	74.07%	Setup & source MMLU-Pro, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (12,032 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.7406914893617021 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 12032 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · BBH fixed-option tasks (accuracy)	93.81%	Setup & source BBH fixed-option tasks, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (5,507 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9380788087888142 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 5507 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · RAGTruth response-level hallucination (F1 on hallucinated class)	86.13%	Setup & source RAGTruth response-level hallucination, F1 on hallucinated class, from Bespoke Labs' own run of the Decision Index 0.3 public suite (2,700 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8612538540596094 on a 0–1 scale (F1 on hallucinated class). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 2700 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · HoVer claim verification (accuracy)	89.35%	Setup & source HoVer claim verification, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (4,000 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8935 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 4000 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · When2Call MCQ (accuracy)	91.37%	Setup & source When2Call MCQ, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (3,652 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.9137458926615553 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 3652 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source
Nimble-v4-max Decision Index runs · New Yorker caption matching (accuracy)	81.82%	Setup & source New Yorker caption matching, accuracy, from Bespoke Labs' own run of the Decision Index 0.3 public suite (528 scored requests). Release v4.1; Bespoke engine NimbleEngine, run nimble-v4.1-max, edition 0.3 scoring. Published as 0.8181818181818182 on a 0–1 scale (accuracy). Bespoke Labs' own Decision Index 0.3 public runs, published in bespokelabs/nimble-v4-max-decision-index (revision 293b569): v4.1 scored 3 November 2026 and v4.0 on 18 October, on 16 NVIDIA H200 GPUs plus 4 for the rebuilt GSM8K rows, 152,955 requests each. Each run's weights_sha256 is the SHA-256 of the weights_sha256.json manifest at that tag's revision. Values are each benchmark's own native metric, published as fractions and shown in percent; the card prints 21 of them rounded to one decimal. Not the community board's chance-corrected index points. Not ranked here. Checked / reported: 2026-11-05 · Samples: 528 Snapshot: 2026-11-03 · Revision: 293b569eb9f46a3e6175e45302abf5fdbe4212dc · Published observations; result reuse terms not specified Result source · Benchmark source

Nimble-v4-max v4.1 · Evaluator-published · Community Decision Index
Evaluation	Result	Evidence
HF text 0.4.1 · Full score	70.03	Setup & source HF text 0.4.1 · Full score Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Public index	72.74	Setup & source HF text 0.4 · Public index Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Private same-skill component	67.39	Setup & source HF text 0.4 · Private same-skill component Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4.1 · Private new-domain component	58.09	Setup & source HF text 0.4.1 · Private new-domain component Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Raw public index	79.51	Setup & source HF text 0.4 · Raw public index Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Public breadth index	72.20	Setup & source HF text 0.4 · Public breadth index Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Equated private same-skill	67.12	Setup & source HF text 0.4 · Equated private same-skill Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4.1 · Equated private new domains	74.39	Setup & source HF text 0.4.1 · Equated private new domains Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Full contribution · public	0.00	Setup & source HF text 0.4 · Full contribution · public Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Full contribution · same_skills	40.27	Setup & source HF text 0.4 · Full contribution · same_skills Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4.1 · Full contribution · new_domains	29.76	Setup & source HF text 0.4.1 · Full contribution · new_domains Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Knowledge · raw	68.83	Setup & source HF text 0.4 · Knowledge · raw Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6883, "skill": 0.6138, "combined": 0.5344, "coverage": 1, "pending": 0, "n": 11} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 11 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Knowledge · skill	61.38	Setup & source HF text 0.4 · Knowledge · skill Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6883, "skill": 0.6138, "combined": 0.5344, "coverage": 1, "pending": 0, "n": 11} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 11 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Knowledge · combined	53.44	Setup & source HF text 0.4 · Knowledge · combined Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6883, "skill": 0.6138, "combined": 0.5344, "coverage": 1, "pending": 0, "n": 11} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 11 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Language · raw	83.44	Setup & source HF text 0.4 · Language · raw Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.8344, "skill": 0.7631, "combined": 0.6419, "coverage": 1, "pending": 0, "n": 9} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 9 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Language · skill	76.31	Setup & source HF text 0.4 · Language · skill Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.8344, "skill": 0.7631, "combined": 0.6419, "coverage": 1, "pending": 0, "n": 9} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 9 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Language · combined	64.19	Setup & source HF text 0.4 · Language · combined Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.8344, "skill": 0.7631, "combined": 0.6419, "coverage": 1, "pending": 0, "n": 9} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 9 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Retrieval · raw	79.07	Setup & source HF text 0.4 · Retrieval · raw Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7907, "skill": 0.7244, "combined": 0.703, "coverage": 1, "pending": 0, "n": 6} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Retrieval · skill	72.44	Setup & source HF text 0.4 · Retrieval · skill Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7907, "skill": 0.7244, "combined": 0.703, "coverage": 1, "pending": 0, "n": 6} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Retrieval · combined	70.30	Setup & source HF text 0.4 · Retrieval · combined Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7907, "skill": 0.7244, "combined": 0.703, "coverage": 1, "pending": 0, "n": 6} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Tools · raw	90.45	Setup & source HF text 0.4 · Tools · raw Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.9045, "skill": 0.8897, "combined": 0.8667, "coverage": 1, "pending": 0, "n": 5} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 5 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Tools · skill	88.97	Setup & source HF text 0.4 · Tools · skill Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.9045, "skill": 0.8897, "combined": 0.8667, "coverage": 1, "pending": 0, "n": 5} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 5 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Tools · combined	86.67	Setup & source HF text 0.4 · Tools · combined Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.9045, "skill": 0.8897, "combined": 0.8667, "coverage": 1, "pending": 0, "n": 5} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 5 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Arts · raw	77.81	Setup & source HF text 0.4 · Arts · raw Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7781, "skill": 0.638, "combined": 0.6293, "coverage": 1, "pending": 0, "n": 6} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Arts · skill	63.80	Setup & source HF text 0.4 · Arts · skill Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7781, "skill": 0.638, "combined": 0.6293, "coverage": 1, "pending": 0, "n": 6} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text 0.4 · Arts · combined	62.93	Setup & source HF text 0.4 · Arts · combined Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.7781, "skill": 0.638, "combined": 0.6293, "coverage": 1, "pending": 0, "n": 6} Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · ECE	7.44 pp	Setup & source HF text calibration 0.4 · ECE Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 34406 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Brier	0.2606 Brier	Setup & source HF text calibration 0.4 · Brier Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 34406 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Sample accuracy	82.50%	Setup & source HF text calibration 0.4 · Sample accuracy Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 34406 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Mean chosen confidence	89.82%	Setup & source HF text calibration 0.4 · Mean chosen confidence Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 34406 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Wrong among answers at ≥95% confidence	3.48%	Setup & source HF text calibration 0.4 · Wrong among answers at ≥95% confidence Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: 34406 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Knowledge · ECE	12.75 pp	Setup & source HF text calibration 0.4 · Knowledge · ECE Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Knowledge · Sample accuracy	73.55%	Setup & source HF text calibration 0.4 · Knowledge · Sample accuracy Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Knowledge · Mean chosen confidence	85.67%	Setup & source HF text calibration 0.4 · Knowledge · Mean chosen confidence Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Language · ECE	3.98 pp	Setup & source HF text calibration 0.4 · Language · ECE Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Language · Sample accuracy	91.21%	Setup & source HF text calibration 0.4 · Language · Sample accuracy Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Language · Mean chosen confidence	95.20%	Setup & source HF text calibration 0.4 · Language · Mean chosen confidence Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Retrieval · ECE	5.61 pp	Setup & source HF text calibration 0.4 · Retrieval · ECE Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Retrieval · Sample accuracy	83.06%	Setup & source HF text calibration 0.4 · Retrieval · Sample accuracy Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source
HF text calibration 0.4 · Retrieval · Mean chosen confidence	87.88%	Setup & source HF text calibration 0.4 · Retrieval · Mean chosen confidence Exact source engine nimble-v4-max-v4.1; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27356728560, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/bespokelabs/Nimble-v4-max", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null, "org": {"id": "bespokelabs", "name": "Bespoke Labs", "logo": "data/orgs/bespokelabs.png"}, "pr": 360, "replaces": "nimble-v4-max", "new": true}. New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Decision Index 0.4.1 (release-v4), source generated 2026-11-04T17:52:26Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-27 PR batch (PR #360). Replaces the earlier board entry nimble-v4-max (newer version of the same model). Submitted public 0.3 run adopted (kit rescore 72.74, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.75% on 891 rows (bar 98.5%). New weights for Nimble v4 Max with the same engine; full-attention layers read the prompt in both directions. A separate model from Bespoke Nimble 9B v3, which stays on the board. License CC BY-NC 4.0. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. Board pull request 360 names v4.1 and replaces the board entry for v4.0; the board names no served checkpoint. The registry pins tag v4.1 (a29d86e), the only published v4.1 weights. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-05 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source